Contactless Respiration Monitoring using Ultrasound Signal with Off-the-shelf Audio Devices

Article (PDF Available) · September 2018with 134 Reads
DOI: 10.1109/JIOT.2018.2877607
Abstract
Recent years have witnessed advances of Internet of Things (IoT) technologies and their applications to enable contactless sensing and elderly care in smart homes. Continuous and real-time respiration monitoring is one of the important applications to promote assistive living for elders during sleep and attracted wide attention in both academia and industry. Most of the existing respiration monitoring systems require expensive and specialized devices to sense chest displacement. However, chest displacement is not a direct indicator of breathing and thus false detection may often occur. In this paper, we design and implement a real-time and contactless respiration monitoring system by directly sensing the exhaled airflow from breathing using ultrasound signals with off-the-shelf speaker and microphone. Exhaled airflow from breathing can be regarded as air turbulence, which scatters the sound wave and results in Doppler effect. Our system works as an acoustic radar which transmits sound wave and detects the Doppler effect caused by breathing airflow. We mathematically model the relationship between the Doppler frequency change and the direction of breathing airflow. Based on this model, we design a Minimum Description Length (MDL) based algorithm to effectively capture the Doppler effect caused by exhaled airflow. We conduct extensive experiments with 25 participants (7 elders, 2 young kids and 16 adults, including 11 females and 14 males) in four different rooms. The participants take four different sleep postures (lying on one’s back, on right/left side and on one’s stomach) in different positions of the bed. Experiment results show that our system achieves a median error lower than 0.3 breaths/min (2%) for respiration monitoring and can accurately identify Apnea. The results also demonstrate that the system is robust to different respiration styles (shallow, normal and deep), respiration rate variation, ambient noise, sensing distance variation (within 0.7 m) and transmitted signal frequency variation.
1
AbstractRecent years have witnessed advances of Internet of
Things (IoT) technologies and their applications to enable
contactless sensing and elderly care in smart homes. Continuous
and real-time respiration monitoring is one of the important
applications to promote assistive living for elders during sleep and
attracted wide attention in both academia and industry. Most of
the existing respiration monitoring systems require expensive and
specialized devices to sense chest displacement. However, chest
displacement is not a direct indicator of breathing and thus false
detection may often occur. In this paper, we design and implement
a real-time and contactless respiration monitoring system by
directly sensing the exhaled airflow from breathing using
ultrasound signals with off-the-shelf speaker and microphone.
Exhaled airflow from breathing can be regarded as air turbulence,
which scatters the sound wave and results in Doppler effect. Our
system works as an acoustic radar which transmits sound wave
and detects the Doppler effect caused by breathing airflow. We
mathematically model the relationship between the Doppler
frequency change and the direction of breathing airflow. Based on
this model, we design a Minimum Description Length (MDL)
based algorithm to effectively capture the Doppler effect caused
by exhaled airflow. We conduct extensive experiments with 25
participants (7 elders, 2 young kids and 16 adults, including 11
females and 14 males) in four different rooms. The participants
take four different sleep postures (lying on one’s back, on
right/left side and on one’s stomach) in different positions of the
bed. Experiment results show that our system achieves a median
error lower than 0.3 breaths/min (2%) for respiration monitoring
and can accurately identify Apnea. The results also demonstrate
that the system is robust to different respiration styles (shallow,
normal and deep), respiration rate variation, ambient noise,
sensing distance variation (within 0.7 m) and transmitted signal
frequency variation.
Index TermsRespiration Detection, Doppler Effect, Acoustic
Sensing, Contactless Sensing.
This work was supported in part by a grant from National Natural Science
Foundation of China (No. 61332013), Chinese Scholarship Council Program.
Tianben Wang and Xinshe Zhou are with the School of Computer Science,
Northwestern Polytechnical University,Xi’an, 710072, China. E-mail:
Daqing Zhang and Bernadette Dorizzi are with the Institut Mines-Técom/
com SudParis 9, rue Charles Fourier 91011 Evry Cedex, France. E-mail:
Leye Wang is with the Department of Computer Science and Engineering,
The Hong Kong University of Science and Technology, Hong Kong, China.
Yuanqing Zheng is with the Department of Computing, The Hong Kong
Polytechnic University, Hong Kong, China. E-mail:
Tao Gu is with the School of Computer Science, RMIT University,
Melbourne VIC 3001, Australia. E-mail: [email protected]t.edu.au.
I. INTRODUCTION
Non-intrusive vital signs monitoring is an important topic for
smart home and smart healthcare [1-4]. Respiration rate is a
vital sign that informs health conditions, indicates progression
towards recovery, and tracks decline of illness. Abnormal
respiratory events such as obstructive or central sleep
apnea-hypopnea are quite common in the elders [25-27]. These
respiration disorders reduce sleep quality and even threaten
one’s life. In particular, chronic obstructive pulmonary disease
(COPD) is the third most common cause of death for people
aged 65 and above [8]. Thus, it is crucial to monitor one’s
respiration continuously and accurately in real-time at home for
elders, especially those living alone, with respiratory diseases.
The traditional way to monitor vital signs requires a person
to visit hospitals or wear dedicated respiration monitoring
devices such as thoracic impedance pneumography [9] or
capnography [10]. However, these methods are quite costly and
also intrusive, preventing these systems from large scale
deployment at home settings. In order to develop cost-effective
and non-intrusive respiration monitoring systems during sleep,
researchers turn their attention to contactless sensing [4-7]. The
approaches based on laser [12], microwave [14], commodity
Wi-Fi [15, 16, 32-38] or acoustic devices [13, 31, 44] to
monitor respiration rate in a contact-free manner. These
approaches typically measure respiration by detecting the
displacement of human chest. However, the chest movement is
hard to measure with current approaches when a user is covered
by a thick blanket or quilt during sleep. In addition, for a user
suffering from obstructive sleep Apnea (OSA), the respiration
may stop (i.e., no exhaled airflow), but the chest can still move
as if the user is breathing normally [28]. As such, current
approaches based on chest movement detection cannot
accurately monitor respirations or reliably detect abnormalities.
Considering the cost and functional requirements of respiration
monitoring in home settings, an ideal respiration monitoring
system should: (1) directly sense breathing airflow, rather than
chest movement, (2) leverage the existing cheap commodity
devices, and (3) be non-intrusive and ideally contact-free.
In this paper, we design and implement a contact-less human
respiration monitoring system using commodity speaker and
microphone. The system directly senses exhaled airflow with a
pair of acoustic transceiver which consists of one speaker and
one microphone. The speaker transmits inaudible sound waves
(i.e., disturb-free design) to be received by the microphone. Our
motivation is based on the observation that exhaled airflow
from breathing causes signal changes in the received sound
waves. Our aim is to detect such changes for respiration
Contactless Respiration Monitoring using Ultrasound
Signal with Off-the-shelf Audio Devices
Tianben Wang, Daqing Zhang, Leye Wang, Yuanqing Zheng, Tao Gu, Bernadette Dorizzi, Xingshe Zhou
2
monitoring. Note that our system does not require users to wear
any devices.
Building an acoustic-based respiration system entails many
practical challenges. First, although exhaled airflow would
cause changes in sound waves, it still remains elusive to
reliably monitor respirations by analyzing the received sound
waves. Besides, without a comprehensive theoretical model to
capture the inherent influences of respirations to the received
sound waves, it is hard to configure the monitoring system to
accurately monitor and reliably detect the sound waves. Second,
a user may change sleep postures and the direction of exhaled
airflow may vary during sleep. It is difficult to quantify the
influence of changing airflow directions. Third, many real-life
factors (e.g., body movement and wind) in the environment
may cause changes in the airflow from breathing, affecting
detection accuracy.
This paper aims to overcome these challenges. We first
conduct several experiments to: 1) study the feasibility of
sensing exhaled airflow leveraging the off-the-shelf acoustic
devices, and 2) investigate the characteristics of sound wave
changes during human breathing. Based on our empirical study,
we then build a theoretical model to describe the relationship
between the variation pattern of Doppler shifts and the direction
of breathing airflow. Based on our theoretical model, we
optimize the system parameters to effectively catch the Doppler
shift to substantially enhance detection performance. We then
profile the Doppler shift using Power Density Spectrum (PSD)
in a specific band derived from the model mentioned above.
Afterwards, to meet the real-time requirement, PSD is
compressed exploiting Minimum Description Length (MDL)
principle [24]. To reduce the dimension of PSD while keeping
sensitivity for exhaled airflow, the compressing method
segments PSD into several bands and ensures that the PSD
segment in the same band has similar sensitivity for exhaled
airflow. Finally, we leverage the periodicity of respiration to
differentiate body movement or other noise factors to further
improve the robustness and accuracy of our system in real
practical scenarios. A demo video is provided at
https://tinyurl.com/ybncm2jz, which verifies the feasibility of
sensing exhaled airflow using commodity acoustic devices
(00:00 - 03:47), illustrates the theoretical model described in
Section III.A (03:48 - 04:57), and records one measurement
study (04:58 - 08:22). The contributions of this paper can be
summarized as follows:
1) We design and implement a respiration monitoring system
which directly senses breathing airflow by leveraging
commodity microphone and speaker.
2) We model the relationship between the exhaled airflow
direction and the Doppler frequency change pattern. Based on
the model, we design an MDL-based compressing algorithm to
effectively capture the Doppler Effect caused by exhaled
airflow.
3) We design an auto-correlation based method to
characterize the periodicity of the Doppler effect and
differentiate respirations from nonperiodic Apnea and body
movement.
4) We conduct extensive experiments to evaluate our system
in four different rooms with 25 participants. The participants
take four different sleep postures (lying on one’s back, on
right/left side and on one’s stomach) in different positions of
the bed. The experiment results show that our system achieves a
median error lower than 0.3 breaths/min (2%) for respiration
monitoring and can accurately identify Apnea. We also conduct
extensive experiments to evaluate system robustness in various
scenarios and the results show that the system is robust to
different respiration styles (shallow, normal and deep),
respiration rate variation, ambient noise, sensing distance
variation (within 0.7 m) and transmitted signal frequency
variation (within the band [20KHz, 21KHz]).
II. RELATED WORK
A. Contact-based Methods
The traditional vital sign monitoring systems require hospital
visits and contact-based monitoring devices. For instance,
thoracic impedance pneumography [9] needs to attach
electrodes on a subject’s chest and measures the change of
electrical impedance during the subject’s respiration.
Capnography [10] utilizes the partial pressure of carbon
dioxide to monitor a subject’s respiration. Both devices need to
be operated by medical specialists in hospitals and clinics and
incur high deployment costs, which are not affordable for large
scale deployment in ordinary homes. Moreover, they require
subjects’ direct contact with the devices and cause
inconvenience for everyday use. Other works [19] adopt
wearable sensing systems and build bed sensing systems with
pressure sensors [20]. However, they still need specialized
devices and are not suitable for large-scale deployment [21].
B. Contactless Method
Compared to the contact-based approaches, the contactless
methods do not require direct contact with monitoring devices.
In the literature, most of the contactless solutions leverage
various signals to detect chest movement during respiration,
such as laser [12], ultrasonic sensors [13] and radio frequency
technologies, like microwave [14], WiFi [15, 16, 32-38] and
RFID [46]. All these works detect respiration by measuring the
chest movement displacement during respiration. However, the
chest movement is hard to measure with current approaches
when a user is covered by thick blanket or quilt during sleeping.
Moreover, these methods assume that chest movement is a
reliable indicator of respiration. However, such an assumption
may not always hold in practice. For instance, the users
suffering obstructive sleep Apnea (OSA) could stop breathing
(i.e., no exhaled airflow) but their chests may still move as if
the users were breathing normally [28]. It can be
life-threatening if a monitoring system considers such chest
movements without exhaled airflows as normal.
Additionally, visual analysis based methods have also been
investigated. For instance, camera-based method [11] utilizes a
Time-of-Flight camera to record subjects’ daily activities and
adopts computer vision algorithms to analyze subjects’
respiration. The main problem is that the camera based methods
may raise users’ privacy concerns. Besides, camera-based
methods highly rely on good lighting conditions.
Acoustic based approaches [53] have recently attracted wide
attention. The method proposed in [30] detects respiration by
recording breathing sound with earphone. However, users may
not be willing to wear earphone when they sleep. The methods
proposed in [31,44] detect respiration by measuring chest
3
displacement during breathing with acoustic signals. As
mentioned above, these methods cannot reliably monitor
respiration if a user is covered by blanket or in case of OSA.
The work [18] requires a specialized device to generate and
receive high frequency (40KHz) ultrasound signals.
Commodity speakers however typically transmit in the
spectrum between 20 Hz to 20 kHz [48]. Moreover, many
existing respiration detection approaches are designed for a
controlled sleep posture, and thus do not work well when users
change their postures during sleep.
III. ACOUSTIC DOPPLER SHIFT CAUSED BY EXHALED
AIRFLOW
Theoretically, the basic principle supporting acoustic
respiration detection is that the intermittent exhaled airflow of
respiration can be seen as turbulence and cause the Doppler
frequency shift. In this section we intend to answer the
following questions: 1) why exhaled airflow incurs acoustic
Doppler shift, and 2) what is the relationship between the
speed and direction of exhaled airflow and the Doppler
shift. In order to answer the above questions, we first derive a
mathematical model to quantify the Doppler frequency shift
caused by exhaled airflow. To verify the derived model, we
conduct real experiments using commodity microphone and
speaker. Finally, we study various factors that may interfere the
acoustic Doppler frequency shift, e.g., body movements, wind,
etc.
A. Acoustic Doppler Shift Caused by Exhaled Airflow
The exhaled airflow can be regarded as turbulence, which is
able to scatter ultrasound signals. As turbulence contains many
unsteady vortices moving irregularly [29], the velocity of
turbulence at time , i.e., is generally composed of two
parts: average velocity and fluctuating velocity , i.e.,
. During breathing, the average velocity of
exhaled airflow mainly contributes to , whose direction and
norm are relatively steady, while the irregularly moving
vortices mainly contribute to , whose direction and norm
change over time. Projecting to the line between the
scatterer and the device, suppose the angle of , and
are , and , respectively, the projection result of
can be denoted as:
 
The traditional Doppler shift [47] is given by
 

Replacing with the projection of , i.e.,
, we can finally derive the Doppler shift
caused by exhaled airflow as:
 


 is positive if the scatterer moves towards the device.
Otherwise,  is negative.
According to our empirical results, the value of
 can be roughly estimated as 0.9 m/s by setting
in Eq. (3) as  
. Then, the value of can be roughly
estimated as 1.3 m/s by setting in Eq. (3) as 0. Referring to
`the above model, we have the following observations. When
angle is close to 0, we have 
  , so  .
So we can only observe the frequency shift above the transmit
frequency . When angle α changes from 0 to  
gradually,
we will observe that the frequency shift above decreases and
the frequency shift below starts occurring. When angle α
stabilizes at around  
, we have 
 , due to the randomness of
in time domain and space, we will observe symmetric
frequency shifts below and above f simultaneously.
The Doppler frequency shift variation when changes from
 
to and the Doppler frequency shift variation when
changes from 0 to  
are symmetric with respect to transmit
frequency .
B. Empirical Verification of Acoustic Doppler Shift Caused by
Exhaled Airflow
In this section we conduct two experiments to verify 1) the
feasibility of sensing exhaled airflow using commodity
acoustic devices, 2) the theoretical model described in Section
III.A
Experimental settings: We bind a speaker (JBL Jembe, 6 Watt,
80 dB) and a microphone (SAMSON Meteor Mic, 16 bit, 48
KHz) as a simple acoustic radar, as shown in Fig. 1(a). The
device is placed in front of a subject facing toward the effective
sensing area where the exhaled airflow passes (as shown in Fig.
1(b)) at a distance of 50 cm. The speaker transmits inaudible
ultrasound waves at  continuously. The speaker
sends ultrasound signals which are scattered in the effective
sensing area due to the exhaled airflow of the subject.
Meanwhile, the microphone records ultrasound signals (with
the sampling rate of 48 KHz with 16 bits).
Experimental protocol: The subject is asked to breathe
naturally first, and then wears a face mask and keeps breathing.
This process is repeated twice. The power density spectrum
(PSD) of the reflected ultrasound signal is shown
synchronously.
Experiment results: The experiment process was recorded in
the demo video provided at the end of Section I.C (the part
00:00 - 03:47). From this experiment we observe: 1) when the
subject breathes naturally without face mask, the echo Power
Density Spectrum (PSD) shows clear periodical amplitude
variation, i.e., micro-Doppler shift around 20KHz with
respiration. Note that the ultrasound signal backward scattered
by exhaled airflow is only a very small part of the echo and its
power is low. The main part of the echo is the ultrasound signal
reflected from the static room environment. This could explain
the reason that even though the signal backward scattered by
exhaled airflow embedded Doppler effect. The main frequency
of the echo keeps the same as transmitted signal at a frequency
of 20KHz. While breathing with face mask, the exhaled airflow
is blocked and the micro-Doppler effect disappears. The
experimental results demonstrate that exhaled airflow can
indeed cause Doppler shifts.
4
Next, we verify our theoretical model and conduct the
following experiments.
Experimental settings: To facilitate the adjustment of the
acoustic beam direction, the acoustic transceiver (as shown in
Fig. 1(a)) is fixed on a tripod facing toward the effective
sensing area where the exhaled airflow passes (as shown in Fig.
1(c)).
Experimental protocol: To verify our theoretical model, we
first collect ultrasound echo signals in the following four
scenarios: breathing at different angles α = 0,  
,  
, and
 
, respectively, where α denotes the angle between the
exhaled airflow direction and the acoustic beam direction. We
then compute the Power Spectrum Density (PSD) of the
received ultrasound echo in each scenario to observe the
micro-Doppler shift.
Experiment results: As shown in Fig. 1(d) ~ (g), the results of
the Doppler shift with different angle match the results
derived from our model. For (Fig. 1(d)), we only
observe a relatively strong frequency shift above transmitting
frequency 20 KHz. For  
(Fig. 1(d)), the frequency
shift above 20KHz reduces. When  
(Fig. 1(e)), the
frequency shift below 20KHz starts to appear. While  
(Fig. 1(f)), we observe almost symmetrical frequency shifts on
both sides of 20KHz. The experiment process was recorded in
the demo video provided at the end of Section I.C (the part
03:48 - 04:57)
Our experiments demonstrate that the Doppler
frequency shift varies with the angle between the exhaled
airflow direction and the acoustic beam direction. With the
angle changes, the frequency shift may appear on one side
or two sides of the transmitting frequency.
C. Other Factors Interfering Acoustic Doppler shift
Other factors may interfere with the acoustic Doppler
frequency shift caused by exhaled airflow. We now discuss two
factors: body movement and wind.
1) Body Movement
The Doppler frequency shift caused by human body
movement is much stronger than the Doppler frequency shift
caused by exhaled airflow. When body movement exists, the
Doppler frequency shift caused by exhaled airflow will be hard
to detect.
2) Wind
The exhaled airflow is the direct detecting target in our
system. When there exists wind in the effective sensing area (as
shown in Fig. 1(b)), the exhaled airflow will be disturbed and
influence the respiration monitoring accuracy.
In reality, when a person sleeps, most of these interference
factors can be eliminated or well controlled. (1) A person will
generally keep stable and her respiration rate can be roughly
estimated while she is moving or turning for a short while
during sleep. To mitigate the problem caused by body
movement, we can suspend respiration detection when body
movement occurs and activate it for detecting respiration after
the body movement disappears. (2) People who need
respiration monitoring could be asked to avoid fans or direct
airflow blowing directly towards their bodies, especially for the
elders and kids.
IV. RESPIRATION DETECTION EXPLOITING PERIODICAL
ACOUSTIC DOPPLER SHIFT
As mentioned in Section III.C, to design a respiration
detection system that is robust to breathing direction variation,
we need to 1) effectively capture the Doppler frequency shift
regardless of the breathing direction, and 2) identify the
Doppler frequency shifts due to respiration from the received
ultrasound signals. We employ Power Spectrum Density (PSD)
to capture the Doppler shift in a specific frequency band
derived from the model in Eq. (3). We notice that human
respiration is generally periodical, while Apnea and the
interferences such as body movement have no periodicity.
Based on this observation, we differentiate respiration from
other arhythmical factors by leveraging the rhythmicity of
human respiration.
(a) (b) (c) (d)
(e) (f) (g)
Fig. 1. The variation of Doppler frequency shift caused by exhaled airflow as radius angle varies. (a) the device we use, (b) the effective sensing area on the body of
subject, (c) panorama of experimental environment (d) radius angle , (e)  
, (f)  
, (g)  
.
Microphone
Speaker
Effective
Sensing Area
Micro-Doppler Shift,
i.e., breathing
5
Based on the above idea, we propose a framework (as shown
in Fig. 2) which consists of three key steps. In the first step, we
profile the Doppler frequency shift using Power Spectrum
Density (PSD). Next, in order to meet real-time requirement,
we compress PSD into a short vector base on the Minimum
Description Length (MDL) principle. Finally, we measure the
periodicity of the elements in the short vector to detect whether
the subject is breathing without body movement or Apnea. If
the subject is detected as breathing without body movement or
Apnea, we locate each breath as the peak of Doppler frequency
shift. Otherwise, we use variance of Doppler shift to
differentiate Apnea and body movement. We now describe the
three key steps in details.
A. Profiling Doppler shift using PSD
Power Spectrum Density (PSD) is an effective tool to profile
the Doppler shift of the echo. We firstly construct a sliding
window to buffer the echo. The sliding window abandons the
obsolete echo sample and accepts the latest one continuously.
Then we use a high-pass filter to filter out the ambient noise.
Afterwards, we compute PSD to profile Doppler effect.
According to our empirical study, the value of average
velocity is about 1.3 m/s, the value of maximum of
fluctuating velocity of exhaled airflow, i.e.,  is
about 0.9 m/s. Based on the model described in Eq. (3), the
theoretical upper bound of Doppler shift caused by exhaled
airflow ( , ) is 257Hz. Due to the randomness of
fluctuating velocity of exhaled airflow in time domain
and space, the upper bound of Doppler shift during 1 second is
about 200Hz in practice. Therefore, we only retain the PSD in
the frequency band []. As our emitted
frequency is configured as an inaudible frequency of 20 KHz,
this range is [19800Hz, 20200Hz].
B. Compressing PSD
For convenience in the following description, we define the
part of echo PSD within the frequency band [19800Hz,
20200Hz] as effective PSD vector.
From the overview of the proposed system (as shown in Fig.
2), we can see that measuring the periodicity of Doppler shift is
the key procedure in step 3. Our intuition to measure the
periodicity of Doppler shift is to measure the periodicity of
each frequency bin (i.e., one element in the PSD vector).
However, the high dimension of PSD vector
1
leads to high
computational overhead. We also observe that PSD values in
some adjacent bins exhibit similar trends, meaning that the
values in adjacent bins can be positively correlated. Based on
these observations, we propose to group the PSD values in
adjacent bins into a frequency band so as to enhance signal
strength and reduce computational overhead. Ideally, the
frequency bins grouped into a frequency bands should exhibit
similar trends (named preciseness requirement), and
meanwhile we want to minimize the total number of frequency
bands (named conciseness requirement).
The sensitivity and informativeness of each frequency bin
can be evaluated with its variance. For example, in our
implementation, the iteration period of respiration detection
system is 0.1 second, i.e., respiration detection system produces
a 300-dimension1 effective PSD vector per 0.1 second. If we
collect the effective PSD vectors when monitoring respiration
for 50 seconds, the system accumulates 50/0.1=500 effective
PSD vectors, i.e. a 500×300 matrix. The sensitivity of ith
frequency bin can be measured as the variance of the ith column
of 500×300 matrix. Larger variance indicates higher sensitivity
and thus more informative in respiration monitoring. For
example, Fig. 3(a) shows a set of effective PSD vectors during
5 minutes while breathing and Fig. 3(b) shows the variance
(highlighted as the red line). Thus, the problem is transformed
to segmenting the variance curve of effective PSD vectors with
respect to the above two objectives, i.e., conciseness and
preciseness.
As a matter of fact, the two requirements are contradictory to
each other. For example, if each bin is placed into an
independent band, preciseness is maximized but conciseness is
minimized. In contrast, if all the bins are grouped into the same
band, conciseness is maximized but preciseness is minimized.
Therefore, we need to find an optimal tradeoff between
conciseness and preciseness. To address this problem, we adopt
the Minimum Description Length (MDL) [24] which allows us
to strike a balance between the two requirements.
In particular, the MDL cost is defined as ,
where is the partition strategy, denote the data; is the
cost of the partition strategy;  is the cost of the data
1
The dimension of effective PSD vector  
.  =
48KHz is sampling rate;   is the width of frequency band [
]. NFFT is number of FFT points when computing PSD. In
our implementation, we set NFFT as the length of data, i.e., .
 = 0.75 second is the buffer length in our system implementation. So
 
 
  
Fig. 2. Overview of the proposed framework
Echo buffering
High-pass filtering
Retaining the PSD
in the frequency
band [f-200, f+200]
Compute PSD with
Welch method
MDL Principle
Constructing PSD
compressing method offline
PSD compressing strategy
Compressed PSD
vector
Buffering Compressed
PSD vector
Measuring the periodicity
of Doppler shift
Breaths and Apnea
detection
Step 1 Step 2 Step 3
Step1: Profiling Airflow using Power Spectrum Density
Step2: Compressing PSD
Step3: Online Respiration Detection
Offline PSDs
during sleep
Apnea
Body
movement
(a) (b)
Fig. 3. Sensitivity of PSD for exhaling. (a) PSDs during breathing, (b)
Variance (red line) as the sensitivity of PSD for exhaling and the blue dotted
lines show an optimum partition employing the method based on MDL
principle.
6
description using partition strategy H [24]. The best partition
strategy is to minimize the MDL cost .
Given:
1) the variance of all the frequency bins in effective PSD
vector during the specific time slot,  ,
where is variance of the point at frequency ;
2) partition strategy : suppose  will be segmented into
bands, and the cut-points are ,
where   .  are in the
ascending order, i.e.,    . The
elements of  in each band  are compressed as:



then, the cost of partition strategy , is defined as:



The cost of the data description using partition strategy ,
 is defined as:





()
We can see that  measures the conciseness and
measures the preciseness. As mentioned previously, we need to
find a segmenting strategy that can minimize the MDL cost (i.e.,
). In practice, the computation cost to find the
optimal segmenting strategy is prohibitively high since we need
to consider every subset of the points in the variance curve.
Therefore, we adopt the approximate method [23] to get an
approximate solution which provides a near-optimal grouping
strategy (approximately 80% optimal) in much shorter time. In
Fig. 3(b), the blue dotted lines show the near-optimal grouping
result obtained using the approximate method.
We evaluate the performance of the grouping method with
the four most common sleep postures (sleeping on one’s back,
on left side, on right side and on one’s stomach). We collect
four datasets, each of which contains data collected under each
of the four sleep postures. We also combine the four datasets to
form a mixed dataset. Then, we conduct the following
experiments to validate the effectiveness of our segmenting
method. The experiment settings in this section is the same as
those in Section III.
Experimental protocol: Firstly, we compute four segmenting
strategies by applying our method to the breathing dataset of all
four sleep postures (lying on one’s back, on right/left side and
on one’s stomach) as well as the mix ed dataset, respectively
(as shown in Table 1 we denote these five segmenting strategies
as Segback, Segleft, Segright, Segstomach and Segmixed). Secondly, we
create three uniform segmenting strategies with the segment
number 8, 10, 12 (as shown in Table 1 we denote them as
Seguniform-8, Seguniform-10, Seguniform-12). Finally, we replace the
above eight segmenting strategies into our framework one by
one, and test the framework performance with the four different
sleep postures.
Experimental results: Table 1 shows the respiration detection
error using different segmenting strategies on different sleep
posture datasets, i.e., Segmixed. We observe that the segmenting
strategy computed by applying our segmenting method to the
mixed breathing dataset achieves smallest error rate, while
other seven segmenting strategies miss part of breaths in some
datasets. The error rate 0.3 (2%) of Segmixed when sleeping on
one’s stomach is caused by the fact that when sleeping on one’s
stomach, the Doppler shift is relatively weak than the Doppler
shift when sleeping on left of right side. Weak Doppler shift
will affect the accuracy of respiration monitoring. Nevertheless,
our method is able to detect the Doppler shift in both sides of
transmitted frequency and finally achieves error rate 2%, which
is accurate enough for many respiration monitoring
applications.
With the best segmenting strategy computed from the mixed
dataset with our method, we compress effective PSD vector as a
vector composed of the median values of all the segments.
C. Online Respiration Detection
It is a common sense that human respiration is usually
rhythmical, i.e., relaxed periodicity, while body movement or
other noise factors discussed in Section III.D are not rhythmical.
Thus, the Doppler frequency shift embedded in echo PSD
caused by respiration will inherit the relaxed periodicity. Hence,
the periodicity of Doppler frequency shift can only occur
during respiration. Therefore, we aim to exploit the periodicity
of Doppler frequency shift to detect human respiration. In
particular, we first measure the periodicity of Doppler
frequency shift variations based on the compressed effective
PSD vector. If the Doppler shift variation shows a strong
periodicity in a specific time window, we can infer that the
subject is breathing normally. Then we locate each breath as the
peak of the Doppler shift. Otherwise, we can conclude that
there is body movement or respiration arrest, i.e., Apnea
occurring. In latter case, we use variance of Doppler shift to
differentiate Apnea and body movement. By using a slide
window, when body movement is detected, the system will
automatically stop respiration detection, and when there is no
movement occurring, the system detects the apnea event.
1) Measuring the Periodicity of Doppler Shift
The periodicity of Doppler frequency shift is reflected in the
variation of the echo PSD. We construct a sliding window to
buffer the compressed effective PSD vector. The sliding
window abandons the obsolete compressed effective PSD
vector and accepts the latest one continuously. Suppose the
TABLE 1.
RESPIRATION DETECTION ERROR USING DIFFERENT SEGMENT STRATEGY
On one’s
back
Left side
Right side
On one’s stomach
Segback
0 (0%)
0 (0%)
4.1 (27.3%)
0.6 (4%)
Segleft
6.8 (45.3%)
0 (0%)
0.3 (2%)
0.3 (2%)
Segright
7.2 (48%)
0 (0%)
0 (0%)
0 (0%)
Segstomach
7.9 (52.7%)
0.3 (2%)
0.6 (4%)
0.3 (2%)
Segmixed
0 (0%)
0 (0%)
0 (0%)
0.3 (2%)
Seguniform_8
6.9 (46%)
7.4 (49.3%)
8.2 (54.7%)
7.9 (52.7%)
Seguniform_10
6.8 (45.3%)
9.2 (61.3%)
8.7 (58%)
8.5 (56.7%)
Seguniform_12
7.4 (49.3%)
7.9 (52.7%)
8.2 (54.7%)
8.1 (54%)
Error unit: breaths/min (percentage)
7
compressed effective PSD vector is -dimention, and the
length of the sliding widow is . The system updates a
matrix in each iteration. We measure the periodicity of each
column employing autocorrelation function. Given the  (
) column , where  is the
element at  row and  column of the matrix, the
autocorrelation function of  column is defined as follows:


 

where and are the expectation and standard deviation of the
 column, respectively. If the column is near periodical
(shown as the green time window in Fig. 4(a), corresponding to
breathing quietly), its autocorrelation function (as shown in Fig.
4(b)) looks like a sinusoid but its amplitude decreases gradually.
If the column fluctuates randomly and significantly (shown as
the black time window in Fig. 4(a), corresponding to body
movement), its autocorrelation function (as shown in Fig. 4(c))
looks like an exponential function with a base smaller than 1. If
the trend of the column is relatively stable (shown as the red
time window in Fig. 4(a), corresponding to Apnea), its
autocorrelation function (as shown in Fig. 4(d)) varies
irregularly.
Based on the characteristics of the autocorrelation function
and our observations, we construct a model to recognize
whether one column of the matrix exhibit strong periodicity. It
can be used to dynamically filter out weak or none periodical
columns and also filter interferences due to body movement as
well as environment noise. In particular, the model works as
follows.
Given the autocorrelations, , of the  column,
indexes of peaks (i.e., local maximums, except for the first peak
whose values is 1) of the are denoted as
,
If  , and
If  
then the  column exbihits strong periodical.  and 
are two thresholds determined as follows. As human respiration
rate typically ranges from 12 breaths/min to 40 breaths/min
(note that, after exercise, human respiration can reach 40
breaths/min), and the length of slide window is 10 seconds in
our experiments. Thus, there should be 2~6.7 breaths in one
sliding window meaning that there are 2~7 peaks in the
autocorrelation results. Thus, in our experiments, we set
 and  to filter out body movements and other
noise
If the number of strong periodical columns is larger than a
specific threshold  , we can infer that the subject is
breathing normally. Otherwise, there is body movement or
respiration arrest, i.e., Apnea happens. In our experiment,
 works well in distinguishing periodic respirations and
nonperiodic body movement and Apnea. In next section, we
describe how to identify normal breathing and Apnea online.
2) Identifying Breaths and Apnea Online
As mentioned above, when we infer that the subject is
breathing normally (i.e., the number of strong periodical
columns is larger  ), the next step is to identify each
breathing. During exhaling, the Doppler frequency shift will
first increase and then decrease. The variation of the Doppler
(a) (b) (c) (d)
Fig. 4. Autocorrelation functions of the features corresponding to breathing, body movement and environment noise. (a) one feature in the 75 seconds time
window, the green, black and red dotted rectangles correspond to the time windows when breathing without movement, body movement occurring and only
environment noise respectively. (b) ~ (d) the autocorrelation function of the feature in green, black and red time window respectively.
Fig. 5. Respiration detection results. The blue curve is the variation of the sum of instances of the periodical columns over time. The small red circles mark all
respirations.
Apnea Apnea
Body
movement Body
movement
Fig. 6. Maximum detectable areas of one transceiver.
35°Acoustic beam
Mattress
Nose
100 cm
8
frequency shift over time can be described as the sum of
instances of the periodical columns (as described in Section
IV.C.1). Thus, the exhaling can be identified as the peak of the
normalized Doppler frequency shift. When we infer that body
movement or Apnea happens (i.e., the number of strong
periodical columns is not larger ), we use the variance of
Dopper shift to identify Apnea. Even though body movement
and Apnea both affect the periodicity of Doppler shift, they
result in very different Doppler shift variation. Body movement
causes drastic and irregular Doppler shift and Apnea causes no
Doppler shift. So, we can use variance of normalized Doppler
shift to identify Apnea. Specifically, if the variance of
normalized Doppler shift in sliding window, whose length is 7
seconds, is larger than specific threshold , then body
movement is detected. Otherwise, Apnea is detected. In our
experiment,  = 0.01 works well in differentiating
Apnea and body movement.
Fig. 5 shows an example of the respiration detection results
in 175 seconds’ time frame. The blue curve is the variation of
the sum of instances of the periodical columns over time. We
can see that the system accurately identifies breaths, marked as
small red circles, body movement (from 27 second to 36, from
109 second to 116) and Apnea (from 63 second to 78, from 143
second to 155). By using slide windows, when body movement
is detected, the system will automatically stop respiration
detection, and when movement stops, the system automatically
resumes the detection.
V. EXPERIMENTAL EVALUATION
In this section, we conduct comprehensive experiments to
evaluate the proposed system. First, we introduce the system
configuration and experiment settings. Then, we briefly
describe the baseline method. We conduct experiments with
25 participants (7 elders, 2 young kids and 16 adults,
including 11 females and 14 males) in four different rooms.
The participants take four different sleep postures (i.e., on
one’s back, on right/left side and on one’s stomach) in
different positions of the bed. We compare our system with
the baseline method in various experiment settings. In addition,
we conduct experiments to test whether the system can identify
Apnea. We also test the system robustness against body
movement, wind, different respiration styles (shallow, normal
and deep), respiration rate variation, ambient noise, sensing
distance variation and transmitted signal frequency variation.
A. System Configuration and Experiment Settings
Theoretically, our design is not limited to COTS
microphones and speakers and should be able to implement
using smartphones. In practice, we face some technical
challenges to implement using smartphone. 1) It is hard to
ensure the that the transmitted acoustic beam passes through the
exhaled airflow when the smartphone is placed on the
nightstand or bed. 2) Even though the acoustic beam passes
through the exhaled airflow, the relatively low power of the
speaker on smartphone cannot ensure that the microphone
receive sufficiently strong echo. Therefore, currently we
implement our respiration monitoring system with commodity
microphones and speakers. Specifically, our system consists of
a pair of commodity speaker and microphone (shown in Fig.
1(a)) which forms an acoustic transceiver. The speaker is
programmed to transmit 20 KHz acoustic wave continuously,
which is inaudible to users. Meanwhile, the microphone
receives the echo at 48 KHz sampling rate and sends it to the
connected laptop for data processing and respiration detection.
For data collection and processing, the transceivers are
connected to two laptops (Thinkpad T450 with Intel Core
i5-5200 CPU, 8G RAM; Dell Latitude E6540 with Intel Core
i7-4800MQ, 4GB RAM). The proposed respiration detection
algorithms are implemented in Matlab and run on each laptop
in real-time.
First of all, we test the maximum detectable areas of one
transceiver. To this end, we ask each participant to lie on a bed
and place one transceiver above the subject’s head for
respiration detection. The experiment settings and
corresponding results are illustrated in Fig. 6. We can see that
the effective maximum detection distance is about 100 cm and
the angle of the detectable area for sleeping on one’s back is
about 35°. We note that one transceiver cannot fully cover all
possible facing directions of a subject. As such, we deploy two
(a) (b) (c)
Fig. 7. The device layout and detectable facing in different sleep postures. (a) sleep on one’s back, (b) sleep on right side and on one’s stomach while the head (the
head contour is highlighted as back dashed line) facing right, (c) sleep on left side and on one’s stomach while the head (the head contour is highlighted as back
dashed line) facing left.
Fig. 8. System settings in real room environment.
50cm
160cm
90cm
Detectable area
Undetectable area
9
transceivers at both sides of a subject to fully cover different
sleeping postures. As shown in Fig. 7, two transceivers are
placed at the upper-left and upper-right of the head,
respectively, facing the effective sensing area (as shown in Fig.
1(b)) with an angle of about 60°. The perpendicular distance
between mattress and device is about 50 cm and the distance
between two pairs of devices is about 160 cm. Fig. 8 shows the
real experiment environment.
According to the bias and limits of agreement of clinical
respiration rate monitoring device [49], our targeted error for
respiration monitoring should be smaller than 1 breaths/min.
This accuracy is not only enough for general respiration
monitoring applications but also can be used for the patients
after surgery [49].
B. Baseline Approach
To the best of our knowledge, the paper in [18] is the only
existing work that attempts to detect the airflow of respiration
using ultrasound signals. Thus, we choose it as the baseline for
comparison. However, the baseline method requires a
specialized device to generate 40 KHz ultrasound signals.
Moreover, the respiration detection approach was designed for
a controlled sleep posture. For fair comparison, 1) the baseline
approach is implemented using the same devices and
deployment manner; 2) we adopt the same optimal parameter
settings and configurations as specified in [18] and fine-tune
the system.
C. System Performance Evaluation
We conduct comprehensive experiments to evaluate our
system in four different rooms with 25 subjects, who have
four different sleep postures in different positions of the
bed and compare the system performance with the baseline
approach. The experiment process was recorded in the demo
video provided at the end of Section I.C (the part 04:58 - 08:22).
In addition, we conduct experiments to test whether the system
can identify Apnea.
1) Evaluation with Different Subjects
We recruit 25 participants (7 elders, 2 young kids and 16
adults, including 11 females and 14 males) to evaluate the
effectiveness of our system. To test the system usability for
users, all participants are asked to set up the system and
properly adjust facing direction of audio transceivers according
to the requirements specified in Section V.A. We set aside 15
minutes for the participants to lie on his/her back so that the
participants really fall asleep before measurements. We then
detect the respiration with each subject for about 2 hours.
During the measurements, two subjects watch the video stream
to record the ground truth manually. Fig. 9(a) shows the CDF of
respiration detection error. We can see that the median
respiration detection error of our approach is 0, while that of the
baseline approach is around 0.9 breaths/min (6%). In addition,
the max error of our system is about 0.6 breaths/min (4%),
while that of the baseline is larger than 2.1 breaths/min (14%).
In addition, to further verify the proposed system, we use
Micro-Movement Sensitive Mattress Sleep Monitoring System
RS-611 (produced by Xinxingyangsheng Technology Co., Ltd.
Bejing, China) to record the ground truth during the
experiments. Two subjects are recruited to evaluate system
performance for about 2 hours. The experimental results show
that the median error of the proposed system is 0.
This experimental results indicate: 1) our system is able to
accurately detect human respiration when the participants sleep
on his/her back, outperforming the baseline; 2) the system is
easy to set up for users in practice.
2) Evaluation with Different Sleep Postures
Except for sleeping on one’s back, lying on one side and
sleeping on one’s stomach are also common sleep postures.
With the same experiment settings, all the 25 subjects are
recruited to evaluate the performance of our system under the
condition of subjects lying on one side and sleeping on one’s
stomach. Fig. 9(b), Fig. 9(c) and Fig. 9(d) show the CDF of
respiration detection error when the subjects are sleeping on the
left side, right side and lying on one’s stomach respectively
(note that here lying on one’s stomach requires the exhaled
airflow not been blocked or covered by the pillow. Otherwise,
the system cannot detect respiration. In general, because of
hypoxia the subject has to change sleep posture if the nose is
covered by the pillow more than 10 seconds). The result shows
that our system outperforms the baseline method with different
sleep postures in terms of detection accuracy. In particular, the
median errors of our system are all 0, while the median errors of
the baseline are 0.3 breaths/min (2%), 0.6 breaths/min (4%) and
1.2 breaths/min (8%), respectively, for different sleep postures.
In addition, the max error of our system can be controlled
(a) (b) (c) (d)
Fig. 9. Respiration detection error of 25 subjects with four different sleep postures. (a) sleep on one’s back, (b) facing left, (c) facing right and (d) sleep on one’s
stomach
Fig. 10. The Respiration detection median error in different positions in bed
10
smaller than 0.6 breaths/min (4%), while the maximum error of
the baseline reaches 2.2 breaths/min (14.7%). This is because
when a subject sleeps on his/her back, the angle between
airflow direction and acoustic beam direction is relatively small.
Thus, the baseline method cannot reliably detect the Doppler
frequency shift. In contrast, our method is able to detect the
Doppler shift in both sides of transmitted frequency. Thus, the
Doppler shift caused by breathing with all four sleep postures
can be well captured.
3) Evaluation with Different Positions on Bed
In the experiment, the subjects are asked to sleep in different
positions of the bed, i.e., left part of the bed, middle part of the
bed and right part of the bed. Fig. 10 shows the CDF of median
respiration detection error when the subjects are sleeping in
different positions of the bed. We can see that our method
outperforms the baseline method and achieves a median
respiration detection error lower than 0.3 breath/min.
4) Evaluation with Different Sleep Environments
We deploy the system in four rooms with different sizes and
layouts. Fig. 11 (a) ~ (d) show the average respiration detection
error CDF in the four test rooms, respectively. The median
detection errors of our system for all four sleep postures in four
test rooms are 0 breaths/min, while the median errors of the
baseline are larger than 0.6 breaths/min (4%). There is no
obvious difference in the four test rooms for both our method
and the baseline. It indicates that our system is not sensitive to
the experiment environment.
5) Apnea Detection Evaluation
Detecting Apnea is an important objective of monitoring
respiration during sleep. Even though body movement and
Apnea both affect the periodicity of Doppler shift, they result in
very different Doppler shift variation. Body movement causes
drastic and irregular Doppler shift and Apnea causes no
Doppler shift. So, we can use the variance of Doppler shift to
identify Apnea.
Constrained by legal issues, we could not test our system
with real Apnea patients in hospitals for now. Instead, we
simulate Central Apnea (CA) and Obstructive Apnea (OA)
following the clinical symptom described as “hold breath for a
while” [31, 39], and simulate Hypopnea event following the
clinical symptom described as “breathing becomes shallow
gradually and then recovers” [31]. We recruit 23 participants
(13 male and 10 female) to test Apnea detection performance.
Each participant is asked to simulate Apnea and generate body
movement 10 times during the 30 minutes’ respiration
monitoring period. Fig. 12(a) and Fig. 12(b) show examples of
Doppler shift variation when Apnea and Hypopnea happen
respectively. The experimental results show that the proposed
system can accurately identify all the simulated Apnea.
D. System Robustness Testing
In this section, we conduct experiments to evaluate the
robustness of our system. Specifically, we test various factors
which may influence the performance of our system including
body movement, wind, different respiration styles (shallow,
normal and deep), respiration rate variation, ambient noise,
sensing distance variation and transmitted signal frequency
variation.
1) Impact of Body Movement
Body movement generates strong but arhythmical Doppler
frequency shift variation. The Doppler frequency shift caused
by exhaled airflow would be submerged. Under this condition,
it is difficult to detect breathing. To reduce the false alarm rate,
our system is designed to suspend respiration detection once
body movement is detected and recover for detecting
respiration after the body movement disappears. 22 subjects are
recruited to test whether our system can actually suspend when
a body movement occurs. We detect subjects’ respiration for
about 30 minutes. During the detection process, the subjects
change sleep postures or move limbs several times. Fig. 13
shows two examples that our system suspends the detection
while body movement occurs, and resumes detection when
movement stops.
2) Impact of Wind
Our system works by sensing the exhaled airflow. If the wind
airflow in the effective sensing area (as shown in Fig. 1(b)) is
large enough, the system cannot work well. To quantitatively
test the impact of wind, we conduct an experiment using fan to
(a) (b) (c) (d)
Fig. 11. Respiration detection error in four rooms of different sizes and layout.
(a) (b)
Fig. 12 Respiration detection when Apnea happens. (a) Apnea, (b) Hypopnea.
Apnea Apnea
Hypopnea
11
generate airflow toward subject’s body. The airflow speed is
measured by a handheld anemometer (thermal anemometer
testo 405-V1). We adjust the airflow speed by adjusting the
distance between the fan and subject. The fan is placed at
different places with different direction towards subject’s head.
20 subjects are recruited to test system performance under
different airflow speed. Fig. 14 shows the respiration detection
median errors with different airflow speed. We can see that
when the indoor airflow speed is higher than 1.5 m/s regardless
of the fan directions towards subject’s head, the system cannot
detect respiration rate accurately. The experiment results imply
that our system is sensitive to indoor airflow. In reality, people
under monitoring conditions always avoid fans or air
conditions blowing directly towards their bodies, especially for
elders and kids.
3) Impact of Different Respiration Styles
Breathing strength will affect the system performance.
Generally, a deeper breath will lead to a lower respiration
detection error. We recruit 21 subjects, including 4 young kids
(8 years old on average), 12 adults (26 years old on average)
and 5 elders (63 years old on average), to test the system
performance when different subjects breathing with different
respiration styles. In the first round, the participants breathe
naturally. In the next two rounds, the subjects are asked to
intentionally control their breath and take relatively shallow
and deep breath, respectively. In each round, the system
monitors their respiration for about 30 minutes. Fig. 15 shows
the respiration detection median error for each subject category
in each round. We can see that when breathing naturally and
deeply, the respiration detection median error for three subject
categories are all smaller than 0.5 breath/min. Even when the
adults breathe gently, the median error of the proposed system
is 0. When young kids and elders breathe gently, the respiration
detection median error reaches 3.2 breaths/min (21.3%) and 0.9
breaths/min (6%), respectively. Too gent breaths not only mean
low velocity, which results in weak Doppler shift (i.e. narrow
frequency shift in PSD of echo), but also scatters backward
weak ultrasound signal which results in low energy (i.e., low
amplitude in PSD of echo). Low energy reflected signals and
weak Doppler shifts will increase the respiration detection error
rate. Fortunately, this problem can be mitigated by decreasing
sensing distance. When we decrease the distance between the
transceiver and subjects to 40 cm, the median error of all
subject categories reduces to 0.
4) Impact of Respiration Rate Change
As our system detects the breathing rate by measuring the
periodicity of Doppler shift, respiration rate change will
weaken the periodicity and may influence our system. In this
work, we use a sliding window method which can discard
the obsolete signals and adapt to the change of respiration
rate. 20 participants are asked to do high intensity exercises
like push-up, which will increase the respiration rate
significantly. Then, we let the participants lie on the bed and
detect their respiration rates until the respiration rates
gradually fall back to a normal level (10 ~ 15 breathing
counts per minute). Fig. 16 shows two examples of
respiration detection results after high intensity exercise.
The blue lines show the measured Doppler frequency shift
and the pink lines track the instantaneous respiration rates.
We can observe that the respiration rates change from about
40 breaths/min to about 15 breaths/min. Our system
accurately tracks both rapid breathing and slow breathing
throughout the process.
Fig. 13. Two examples of respiration detection process when body movement
occurs
Fig. 14. Respiration detection error as the speed of interfering airflow varies
Body
movement Body
movement
Body
movement Body
movement
Fig. 15. Respiration detection error when subjects breathe with different
respiration styles
Fig. 16 Two examples of respiration detection process as respiration rate is
changing
12
5) Impact of Ambient Noise
The proposed system senses human respiration using
ultrasound signal. The ambient noise will also be received by
the system. It is necessary to test whether the ambient noise has
impact on respiration detection. We test the proposed system in
several typical real scenes that continues to generate noise.
Specifically, the real scenes include: 1) talking in low voice; 2)
talking in normal voice; 3) talking in loud voice; 4) play music
or video; 5) noise from air condition. 20 subjects are recruited
to test system performance in the above five scenes. Table 2
shows the experimental results. We can see that the typical
ambient noise has no impact on the proposed system. The
proposed system senses exhaled airflow using 20KHz
ultrasound signal and capture the Doppler effect in the
frequency band [19.8KHz, 20.2KHz]. In real scenes, there is
hardly any ambient noise which can reach such a high
frequency band. Studies show that the highest frequency of
human voice is 3KHz [40]; the highest frequency of music is
16KHz [41]; the highest frequency of the noise produced by air
condition or other household electric appliances is 8KHz
[42,43]. The frequencies of all these ambient noises are far
below the system working frequency band, hence, the ambient
noise can be easily filtered out using lowpass filter.
6) Impact of Sensing Distance
We vary the distance between transceiver and subject,
ranging from 0.3 m to 1.1 m with an interval of 0.1 m. 21
subjects are recruited to evaluate the performance of our system.
At each position, we test for 30 minutes. Fig. 17 shows the
median respiration detect error as distance varies. We can see
that within 0.7 m, the system achieves respiration detection
error smaller than 0.5 breaths/min (3.3%). Beyond 0.7 m, the
error will increase with the distance mainly due to signal
attenuation. We hence suggest setting the distance between
transceiver and subject to a value smaller than 0.7 m. In practice
user can also make a tradeoff between respiration detection
error and sensing distance for a specific application
environment.
7) Impact of Transmitted Signal Frequency
The frequency of the transmitted signal should be higher than
the upper bound of human audibility range, 20KHz, and lower
than the upper limit of frequency response of commodity
acoustic device 22KHz. We vary the transmitted signal
frequency from 20KHz to 22KHz with an interval of 0.5KHz.
23 subjects are recruited to performance of our system for about
10 minutes. Fig. 18 shows the median respiration detection
error as the transmitted signal frequency varies. We can see that
within 21KHz, the system achieves respiration detection error
smaller than 1 breaths/min (6.7%). Beyond 21KHz, the error
will increase with the transmitted signal frequency mainly due
to the decrease of frequency response of commodity audio
system. For commodity audio device, beyond 21KHz, the
system frequency response will decrease dramatically. In our
system, we set the transmitted signal frequency to 20KHz. In
practice user can also make a tradeoff between respiration
detection error and transmitted signal frequency for a specific
application environment.
In summary, the proposed system is robust to different
respiration styles (shallow, normal and deep), respiration rate
variation, ambient noise, sensing distance variation (within 0.7
m) and transmitted signal frequency variation (within the band
[20KHz, 21KHz]), but sensitive to wind and body movement.
The experimental results are summarized in Table 3.
E. Discussion
The experiment results demonstrate that the proposed system
can detect human respiration with four common sleep postures
in different positions of the bed. The proposed system is robust
to different respiration styles (shallow, normal and deep),
respiration rate variation, ambient noise, sensing distance
variation (within 0.7 m) and the transmitted signal frequency
variation (within the band [20KHz, 21KHz]). Yet, we note that
current implementation can be improved in the following
aspects:
Body movement: As presented in Sections V.D, the
system is sensitive to sporadic body movement during sleep
and the airflow around the subject. When body movement
occurs, the weak Doppler shift caused by exhaled airflow
would be submerged by the drastic and irregular Doppler shift
caused by body movement; when interference airflow exists in
the effective sensing area, the exhaled airflow will be disturbed.
We plan to detect and filter out Doppler shifts caused by body
movement in the future.
Multiple users: Current system can only be used to
monitor a single person lying on the bed at this moment. With
multiple persons lying on the bed, the exhaled airflow may be
blocked by other persons. In order to simultaneously monitor
multiple people, we plan to study the feasibility of using motors
to adjust acoustic transceivers in the future.
Apnea detection: We note that due to legal issues, we
could not evaluate our system with real Apnea patients in
TABLE 2
RESPIRATION DETECTION ERROR WITH DIFFERENT AMBIENT NOISE
Noise Source
Error (breaths/min, (percentage))
Talking in low voice
0 (0%)
Talking in normal voice
0.3 (2%)
Talking in loud voice
0 (0%)
Play music/video
0.1 (0.67%)
Noise from air condition
0 (0%)
Fig. 17 Respiration detect error as sensing distance is varied
Fig. 18 Respiration detect error as the frequency of transmitted acoustic signal
is varied
13
hospitals at this moment. The performance evaluation of Apnea
detection was conducted by simulating the typical symptoms of
Apnea. To better evaluate the efficacy of Apnea detection, we
plan to conduct more extensive evaluations by inviting real
Apnea patients to our lab in future work.
VI. CONCLUSION AND FUTURE WORK
This paper presents a continuous and real-time respiration
monitoring system that is built purely using commodity audio
devices. It utilizes the Doppler Effect generated by the exhaled
airflow of breath on the acoustic wave as the respiration
indicator. We formally model the relationship between the
exhaled airflow direction and the Doppler frequency change
pattern. Based on this model, we design a MDL-based
algorithm to effectively capture the Doppler effect caused by
exhaled airflows. We implement a practical respiration
monitoring system with commodity microphone and speaker
and tested the detection performance in various experiment
settings. The extensive experiments demonstrate that 1) our
proposed system achieves low respiration detection error
(lower than 0.3 breaths/min (2%)) without assuming subject
sleeping postures and positions in the bed and can accurately
identify Apnea, and 2) our proposed system is robust to
different respiration styles (shallow, normal and deep),
respiration rate variation, ambient noise, sensing distance
variation (within 0.7 m) and transmitted signal frequency
variation. In order to further enhance the robustness of
respiration monitoring performance, we plan to improve the
system so that it can mitigate the influences caused by sporadic
body movement during sleep. We also plan to further evaluate
our system in larger scale deployment in ordinary homes.
ACKNOWLEDGMENT
The authors would like to express their special appreciation
to the 25 volunteers for participating in our experiments. The
study was approved by the Medical Ethics Committee (MEC)
of NPU. This work was supported in part by a grant from
National Natural Science Foundation of China (No. 61332013),
Chinese Scholarship Council Program, National Natural
Science Foundation of China (No. 61702437), and Hong Kong
ECS under Grant PolyU 252053/15E
REFERENCES
[1] Catarinucci, Luca, et al. "An IoT-Aware Architecture for Smart
Healthcare Systems." IEEE Internet of Things Journal
2.6(2015):515-526.
[2] Amendola, S, et al. "RFID Technology for IoT-Based Personal
Healthcare in Smart Spaces." IEEE Internet of Things Journal
1.2(2014):144-152.
[3] Zhang, Yuan, et al. "Ubiquitous WSN for Healthcare: Recent Advances
and Future Prospects." IEEE Internet of Things Journal
1.1(2014):311-318.
[4] Graham D, Simmons G, Nguyen D T, et al. A Software-Based Sonar
Ranging Sensor for Smart Phones[J]. IEEE Internet of Things Journal,
2015, 2(6):479-489.
[5] Gu, Yu, F. Ren, and J. Li. "PAWS: Passive Human Activity Recognition
Based on WiFi Ambient Signals." IEEE Internet of Things Journal
3.5(2017):796-805.
[6] Mahmud, Md. Shaad, et al. "A Wireless Health Monitoring System Using
Mobile Phone Accessories." IEEE Internet of Things Journal
99(2017):1-1.
[7] Gu, Yu, et al. "MoSense: A RF-based Motion Detection System via
Off-the-Shelf WiFi Devices." IEEE Internet of Things Journal
99(2017):1-1.
[8] Madeline R. Vann, MPH, “The 15 Most Common Health Concerns for
Seniors”. URL: http://goo.gl/EQn2fn, 2015
[9] Wilkinson, J. N. and Thanawala, V. U. (2009), Thoracic impedance
monitoring of respiratory rate during sedation is it safe?. Anaesthesia,
64: 455456.
[10] Jaffe, Michael B. "Infrared measurement of carbon dioxide in the human
breath:“breathe-through” devices from Tyndall to the present day."
Anesthesia & Analgesia 107.3 (2008): 890-904.
[11] Penne, Jochen, et al. "Robust real-time 3D respiratory motion detection
using time-of-flight cameras." IJCARS 3.5 (2008): 427-431.
[12] Kondo, T., et al. "Laser monitoring of chest wall displacement." ERJ 10.8
(1997): 1865-1869.
[13] Min, Se Dong, et al. "Noncontact respiration rate measurement system
using an ultrasonic proximity sensor." IEEE Sensors Journal 10.11 (2010):
1732-1739.
[14] Nowogrodzki, M., D. D. Mawhinney, and H. F. Milgazo. "Non-invasive
microwave instruments for the measurement of respiration and heart
rates." NAECON (1984): 958-960.
[15] Liu, Xuefeng, et al. "Wi-Sleep: Contactless sleep monitoring via WiFi
signals." RTSS (2014): 346-355.
[16] Ravichandran, Ruth, et al. "Wibreathe: Estimating respiration rate using
wireless signals in natural settings in the home." PerCom, 2015:131-139.
[17] Venkatesh, Swaroop, et al. "Implementation and analysis of
respiration-rate estimation using impulse-based UWB." MILCOM
2005-2005 IEEE Military Communications Conference. IEEE, 2005.
[18] Arlotto, Philippe, et al. "An ultrasonic contactless sensor for breathing
monitoring." Sensors 14.8 (2014): 15371-15386.
[19] Paradiso, Rita. "Wearable health care system for vital signs monitoring."
EMBS (2003):283-286.
TABLE 3
EXPERIMENTAL RESULTS SUMMARY
Test Category
Test items
Results
Performance
evaluation
Respiration monitoring with different subjects
Median error: 0; Max error: 0.6 (unit: breaths/min)
Respiration monitoring with different sleep postures
Median error: 0; Max error: 0.6 (unit: breaths/min)
Respiration monitoring with different positions on the bed
Median error of left part: 0, middle part: 0.3, right part: 0.1 (unit: breaths/min)
Respiration monitoring with different sleeping environments
Median error: 0; Max error: 0.8 (unit: breaths/min)
Respiration monitoring with Apnea
Accurately identifies all the simulated Apnea
Robustness
testing
Effect of body movement
The system accurately identifies body movement; suspends the detection while
body movement occurs, and resumes detection when movement stops.
Effect of wind
If airflow speed is higher than 1.5 m/s, the system cannot monitor respiration
accurately.
Effect of different respiration styles
When breathing naturally and deeply, the respiration detection median error are
smaller 0.5 breath/min; when young kids and elders breathe gently, median error
reaches 3.2 breaths/min and 0.9 breaths/min, respectively.
Effect of respiration rate change
The system is robust to respiration rate change.
Effect of ambient noise
The system is robust to common ambient noise.
Effect of sensing distance
Max error is smaller than 0.5 breaths/min (distance < 0.7 m)
Effect of transmitting signal frequency
Max error is smaller than 1 breaths/min (frequency range [20KHz, 21KHz])
14
[20] Nukaya, Shoko, et al. "Noninvasive bed sensing of human biosignals via
piezoceramic devices sandwiched between the floor and bed." IEEE
Sensors journal 12.3 (2012): 431-438.
[21] Gokalp, Hulya, and Malcolm Clarke. "Monitoring activities of daily
living of the elderly and the potential for its use in telecare and telehealth:
A review." TELEMEDICINE and e-HEALTH 19.12 (2013): 910-923.
[22] Welch, Peter D. "The use of fast Fourier transform for the estimation of
power spectra: A method based on time averaging over short, modified
periodograms." IEEE Transactions on audio and electroacoustics 15.2
(1967): 70-73.
[23] Lee, Jae-Gil, Jiawei Han, and Kyu-Young Whang. "Trajectory clustering:
a partition-and-group framework." SIGMOD (2007):593-604.
[24] Grünwald, Peter D., In Jae Myung, and Mark A. Pitt. Advances in
minimum description length: Theory and applications. MIT press, 2005.
[25] Cooke, Jana R., and Sonia Ancoli-Israel. "Normal and abnormal sleep in
the elderly." Handbook of clinical neurology/edited by PJ Vinken and
GW Bruyn 98 (2011): 653.
[26] Norman, Daniel, and José S. Loredo. "Obstructive sleep apnea in older
adults." Clinics in geriatric medicine 24.1 (2008): 151-165.
[27] Lee-Chiong, Teofilo L. "Monitoring respiration during sleep." Clinics in
chest medicine 24.2 (2003): 297-306.
[28] Wikipedia: https://en.wikipedia.org/wiki/Obstructive_sleep_apnea
[29] Tennekes, Hendrik, and John Leask Lumley. A first course in turbulence.
MIT press, 1972.
[30] Ren, Yanzhi, et al. "Fine-grained sleep monitoring: Hearing your
breathing with smartphones." INFOCOM (2015):1194-1102.
[31] Nandakumar, Rajalakshmi, Shyamnath Gollakota, and Nathaniel Watson.
"Contactless sleep apnea detection on smartphones."Mobisys
(2015):45-57.
[32] Hao Wang, Daqing Zhang, Junyi Ma, Yasha Wang, Yuxiang Wang, Dan
Wu, Tao Gu, Bing Xie. "Human respiration detection with commodity
wifi devices: do user location and body orientation matter?." Proceedings
of the 2016 ACM International Joint Conference on Pervasive and
Ubiquitous Computing. ACM (2016): 25-36.
[33] Abdelnasser, Heba, Khaled A. Harras, and Moustafa Youssef.
"UbiBreathe: A ubiquitous non-invasive WiFi-based breathing
estimator." Proceedings of the 16th ACM International Symposium on
Mobile Ad Hoc Networking and Computing. ACM, (2015): 277-286
[34] Kaltiokallio, Ossi, et al. "Non-invasive respiration rate monitoring using a
single COTS TX-RX pair." Information Processing in Sensor Networks,
IPSN-14 Proceedings of the 13th International Symposium on. IEEE
(2014): 59-69.
[35] Xuefeng Liu, Jiannong Cao, Shaojie Tang, Jiaqi Wen, and Peng Guo.
“Contactless Respiration Monitoring via WiFi Signals.” IEEE
Transactions on Mobile Computing (2016).
[36] Patwari, Neal, et al. "Breathfinding: A wireless network that monitors and
locates breathing in a home." IEEE Journal of Selected Topics in Signal
Processing 8.1 (2014): 30-42.
[37] Patwari, Neal, et al. "Monitoring breathing via signal strength in wireless
networks." IEEE Transactions on Mobile Computing 13.8 (2014):
1774-1786.
[38] Wu, Chenshu, et al. "Non-invasive detection of moving and stationary
human with wifi." IEEE Journal on Selected Areas in Communications
33.11 (2015): 2329-2342.
[39] WebMD.
http://www.webmd.com/sleep-disorders/guide/sleep-disorders-symptom
s-types.
[40] Human Voice Frequency Range.
http://www.seaindia.in/blog/human-voice-frequency-range/
[41] http://www.liutaiomottola.com/formulae/freqtab.htm
[42] Susini, Patrick, et al. "Characterizing the sound quality of
air-conditioning noise ." Applied Acoustics 65.8(2004):763-790.
[43] Shih-Pin HUANG, Rong-Ping LAI. "FREQUENCY
CHARACTERISTICS OF INTERIOR NOISES IN HOUSES." The 2005
World Sustainable Building Conference,Tokyo, 27-29 September 2005
(SB05Tokyo)
[44] Tianben Wang, Daqing Zhang, Yuanqing Zheng, Tao Gu, Xingshe Zhou,
Bernadette Dorizzi. "C-FMCW Based Contactless Respiration Detection
Using Acoustic Signal." Proceedings of the ACM on Interactive, Mobile,
Wearable and Ubiquitous Technologies 1.4 (2017)
[45] Nguyen, Phuc, et al. "Continuous and fine-grained breathing volume
monitoring from afar using wireless signals." IEEE INFOCOM 2016 - the,
IEEE International Conference on Computer Communications IEEE,
2016:1-9.
[46] Hou, Yuxiao, Y. Wang, and Y. Zheng. "TagBreathe: Monitor Breathing
with Commodity RFID Systems." IEEE, International Conference on
Distributed Computing Systems IEEE, 2017:40-413.
[47] Carrara, Walter G, R. S. Goodman, and R. M. Majewski. "Spotlight
synthetic aperture radar: Signal processing algorithms." Journal of
Atmospheric and Solar-Terrestrial Physics 59.5(1995):597-598.
[48] Guoming Zhang, Chen Yan, Xiaoyu Ji, Taimin Zhang, Tianchen Zhang,
Wenyuan Xu. "DolphinAtack: Inaudible Voice Commands". ACM
Conference on Computer and Communications Security (CCS) 2017
[49] Gaucher, A, et al. "Accuracy of respiratory rate monitoring by
capnometry using the Capnomask(R) in extubated patients receiving
supplemental oxygen after surgery. " British Journal of Anaesthesia
108.2(2012):316-320.
[50] Hagargund, Asha G, and R. R. N. Udayshankar. "Radar Based Cost
Effective Vehicle Speed Detection Using Zero Cross Detection."
International Journal of Electrical Electronics & Data Communication
1.9(2013).
[51] Dybedal, Joacim. "Doppler Radar Speed Measurement Based On A 24
GHz Radar Sensor." Institutt for Elektronikk Og Telekommunikasjon
(2013).
[52] Yolanda, Parra, T. Guzmán, and J. T. Gonzales. "Development of a
Low-Cost, Short-Range Radar System to Measure Speed and Distance."
Tecciencia 12(2017).
[53] Bisio, Igor, et al. "Ultrasounds-based Context Sensing Method and
Applications over the Internet of Things." IEEE Internet of Things
Journal PP.99:1-1.
Tianben Wang received the B.S. degree in
computer science from Northwest A&F
University, Yangling, China, in 2011 and
M.S. degree in computer application
technology from Northwestern
Polytechnical University, Xi’an, China, in
2013, where he is currently working toward
the Ph.D. degree.
His research interests include
ubiquitous computing, contactless behavior
sensing, intelligent elder assisting technology.
Daqing Zhang received the PhD degree
from the University of Rome La
Sapienza in 1996. He is a chair professor
in the School of E ECS, Peking
University, China. He has published more
than 20 0 technical papers in leading
conferences and journals. He served as
the general or program chair for more
than 10 international conferences, giving
keynote talks at more than 16 international conferences. He is
the associate editor for the ACM Transactions on Intelligent
Systems and Technology, IEEE Transactio ns on Big Data, etc.
He is the winner of the 10-years CoMo Rea Impact Paper
Award at the IEEE Per-Com 2013, the Honorable Mention
Award at the ACM UbiComp 2015, the Best Paper Award at
the IEEE UIC 2015 and 2012, and the Best Paper Runner Up
Award at Mobiquitous 2011. His research interests include
context-aware computing, urban computing, mobile computing,
big data analytics, pervasive elderly care, etc. He is a member
of the IEEE.
15
Leye Wang received the B.Sc. and M.Sc.
degrees in computer science from Peking
University, Beijing, China, the Ph.D.
degree from Institut
Mines-Técom/Técom SudParis, Evry,
France, and the Université Pierre et Marie
Curie, Paris, France. He is currently a
senior researcher at the Department of
Computer Science and Engineering, Hong
Kong University of Science and Technology.
His research interests include mobile crowdsensing and
ubiquitous computing.
Yuanqing Zheng received the B.S. degree
in Electrical Engineer ing and the M.E.
degree in Communication and Information
System from Beijing Normal University,
Beijing, China, in 2007 and 2010
respectively. He received the PhD degree i
n School of Computer Engineering from
Nanyang Technological University in
2014. He is currently an Assistant
Professor with the Department of Computing in Hong Kong
Polytechnic University. His research interest includes mobile
and wireless computing and RFID. He is a member of IEEE and
ACM.
Tao Gu received his Bachelor degree
from Huazhong University of Science and
Technology, M.Sc. from Nanyang
Technological University, Singapore, and
Ph.D. in computer science from National
University of Singapore. is currently an
Associate Professor in Computer Science
at RMIT University, Australia. His current
research interests include mobile computing,
ubiquitous/pervasive computing, wireless sensor networks,
distributed network systems, sensor data analytics, cyber
physical system, Internet of Things, and online social networks.
He is a Senior Member of IEEE and a member of ACM.
Bernadette Dorizzi received the Ph.D.
(Thèse d’état) degree in theoretical
physics from the University of Orsay
(Paris XI-France), in 1983, with a focus on
integrability of dynamical systems. She
led the Electronics and Physics
Department from 1995 to 2009. She has
been a Professor with Telecom SudParis
(ex INT) since 1989, and the Dean of
Research since 2013. She has coordinated the Biosecure
Network of Excellence, and is currently the Chairwoman of the
Biosecure Foundation. She is in charge of the Intermedia
(Interaction for Multimedia) Research Team. She has authored
over 300 research papers and has supervised over 20 Ph.D.
thesis.
Her research domain is related to pattern recognition and
machine learning applied to activity detection,
surveillance-video, and biometrics.
Xingshe Zhou received the M.S. degree
from Northwestern polytechnical
University, Xi’an, China, in 1984.
He is a Professor with the School of
Computer Science, Northwestern
Polytechnical University, Xi’an, China.
He is the Director with Shaanxi Key
Laboratory of Embedded System
Technology, Xi’an. His research interests
include embedded computing and pervasive computing.
  • Article
    We present a multi-frequency feature set to detect driver’s 3D head and torso movements from fluctuations in the Radio Frequency (RF) channel due to body movements. Current features used for movement detection are based on time-of-flight, received signal strength and channel state information, and come with the limitations of coarse tracking, sensitivity towards multi-path effects and handling corrupted phase data, respectively. There is no standalone feature set which accurately detects small and large movements and determines the direction in 3D space. We resolve this problem by using two radio signals at widely separated frequencies in a monostatic configuration. By combining information about displacement, velocity and direction of movements derived from the Doppler Effect at each frequency, we expand the number of existing features. We separate Pitch, Roll and Yaw movements of head from torso and arm. The extracted feature set is used to train a K-Nearest Neighbor classification algorithm which could provide behavioral awareness to cars while being less invasive as compared to camera-based systems. The training results on data from 4 participants reveal that at 1.8GHz, the classification accuracy is 77.4%, at 30GHz it is 87.4%, and multi-frequency feature set improves the accuracy to 92%.
  • Article
    Nowadays Internet of Things (IoT) devices can collect a large amount of data and infer the context they are operating in. One of the most pervasive IoT device is the smartphone. Among the plethora of sensors that such device has, microphone is probably the most versatile. It can be used to infer information about the context by acquiring the environmental sound. In this paper we propose an active ultrasonic-based method able to sense context information. The approach is based on the emission of periodic ultrasonic impulses, called pings, whose echoes are continuously acquired to extract a proper set of features. Using a classifier the context information is then retrieved. Two applications are presented: i) an Indoor/Outdoor Detector (IOD), and ii) an Earphone Wearing State Detector (EWSD). In both cases a smartphone is the employed device. The former senses if the smartphone is in an indoor or outdoor environment while the latter detects if a user is wearing or not his earphones. The obtained results are encouraging for both the solutions that have been stressed in different working conditions and employed within proper application frameworks.
  • Conference Paper
    Full-text available
    Recent advances in ubiquitous sensing technologies have exploited various approaches to monitoring vital signs. One of the vital signs is human respiration which typically requires reliable monitoring with low error rate in practice. Previous works in respiration monitoring however either incur high cost or suffer from poor error rate. In this paper, we propose a correlation based ranging method C-FMCW which is able to achieve high ranging resolution. Based on C-FMCW, we present the design and implementation of an audio-based highly-accurate system for human respiration monitoring, leveraging on commodity speaker and microphone widely available in home environments. The basic idea behind the audio-based method is that when a user is close to a pair of speaker and microphone, body movement during respiration causes periodic audio signal changes, which can be extracted to obtain the respiration rate. However, several technical challenges exist when applying C-FMCW to detect respiration with commodity acoustic devices. First, the sampling frequency offset between speakers and microphones if not being corrected properly would cause high ranging errors. Second, the uncertain starting time difference between the speaker and microphone varies over time. Moreover, due to multipath effect, weak periodic components due to respiration can easily be overwhelmed by strong static components in practice. To address those challenges, we 1) propose an algorithm to compensate dynamically acoustic signal and counteract the offset between speaker and microphone; 2) co-locate speaker and microphone and use the received signal without reflection (self-interference) as a reference to eliminate the starting time difference; and 3) leverage the periodicity of respiration to extract weak periodic components with autocorrelation. Extensive experimental results show that our system accurately detects respiration in real environment with the median error lower than 0.35 breaths/min, outperforming the state-of-the-arts.
  • Article
    Motion is a critical indicator of human presence and activities. Recent developments in the field of indoor motion detection have revealed their potentials in enhancing our living experiences through applications like intrusion detection and sleep monitoring. However, existing solutions still face several critical downsides such as the availability (specialized hardware), reliability (illumination and line-of-sight constraints) and privacy issues (being watched). To overcome such shortages, a radio frequency (RF) based device-free motion detection system (MoSense) is designed via leveraging the attenuation of ubiquitous WiFi signals induced by motions to deliver a reliable and transparent detection service in realtime. The design and implementation of MoSense face two challenges: characterizing stationary states and the noisy subcarriers. For the first challenge, a !0silence!1 analysis model is proposed to characterize stationary states for distinguishing motions. For the second challenge, we design a distance-based mechanism to select certain subcarriers that better capture the impact of motions from the noisy channel through measuring the similarity between subcarriers. A prototype of MoSense is realized and evaluated in real environments. By comparing MoSense with other two state-of-the-art systems, i.e., FIMD and FRID, we have shown that MoSense is superior in terms of precision, false negative rate and computational complexity. Considering that MoSense is compatible with existing WiFi infrastructure, it constitutes a low-cost yet promising solution for motion detection.
  • Conference Paper
    Full-text available
    Speech recognition (SR) systems such as Siri or Google Now have become an increasingly popular human-computer interaction method, and have turned various systems into voice controllable systems(VCS). Prior work on attacking VCS shows that the hidden voice commands that are incomprehensible to people can control the systems. Hidden voice commands, though hidden, are nonetheless audible. In this work, we design a completely inaudible attack, DolphinAttack, that modulates voice commands on ultrasonic carriers (e.g., f > 20 kHz) to achieve inaudibility. By leveraging the nonlinearity of the microphone circuits, the modulated low frequency audio commands can be successfully demodulated, recovered, and more importantly interpreted by the speech recognition systems. We validate DolphinAttack on popular speech recognition systems, including Siri, Google Now, Samsung S Voice, Huawei HiVoice, Cortana and Alexa. By injecting a sequence of inaudible voice commands, we show a few proof-of-concept attacks, which include activating Siri to initiate a FaceTime call on iPhone, activating Google Now to switch the phone to the airplane mode, and even manipulating the navigation system in an Audi automobile. We propose hardware and software defense solutions. We validate that it is feasible to detect DolphinAttack by classifying the audios using supported vector machine (SVM), and suggest to re-design voice controllable systems to be resilient to inaudible voice command attacks.
  • Article
    This paper presents the design and prototype of a wireless health monitoring system using mobile phone accessories. We focus on measuring real time Electrocardiogram (ECG) and Heart rate monitoring using a smartphone case. With the increasing number of cardiac patients worldwide, this design can be used for early detection of heart diseases. Unlike most of the existing methods that use an optical sensor to monitor heart rate, our approach is to measure real time ECG with dry electrodes placed on smartphone case. The collected ECG signal can be stored and analyzed in real time through a smartphone application for prognosis and diagnosis. The proposed hardware system consists of a single chip microcontroller (RFduino) embedded with Bluetooth low energy (BLE), hence miniaturizing the size and prolonging battery life. The system called "Smart Case" has been tested in a lab environment. We also designed a 3D printed smartphone case to validate the feasibly of the system. The results demonstrated that the proposed system could be comparable to medical grade devices.
  • Conference Paper
    Recent research has demonstrated the feasibility of detecting human respiration rate non-intrusively leveraging commodity WiFi devices. However, is it always possible to sense human respiration no matter where the subject stays and faces? What affects human respiration sensing and what's the theory behind? In this paper, we first introduce the Fresnel model in free space, then verify the Fresnel model for WiFi radio propagation in indoor environment. Leveraging the Fresnel model and WiFi radio propagation properties derived, we investigate the impact of human respiration on the receiving RF signals and develop the theory to relate one's breathing depth, location and orientation to the detectability of respiration. With the developed theory, not only when and why human respiration is detectable using WiFi devices become clear, it also sheds lights on understanding the physical limit and foundation of WiFi-based sensing systems. Intensive evaluations validate the developed theory and case studies demonstrate how to apply the theory to the respiration monitoring system design.
  • Thesis
    Full-text available
    This thesis will present the implementation of an on-board speed measurement system using a single 24.1 GHz Doppler radar sensor and specialized algorithms to measure the true speed of a vehicle. Two different algorithms are implemented, the first based on estimating the Doppler power density spectrum and extracting the strongest frequency component, and the second based on correlation between the Doppler spectrum and pre-estimated theoretical spectra. The output can be displayed to the user in real-time as well as stored for future reference. An ARM Cortex M4 microcontroller with digital signal processing capabilities is used as the hardware platform, with an audio CODEC chip used as the analog to digital converter. The software is implemented using the C programming language. The system is tested and the measurements are compared to a GPS reference system, with results showing statistical mean errors down to as little as 0.03 km/h and -0.18 %, and a standard deviation of 0.87 km/h during the final test runs.