Conference PaperPDF Available

Improving convulsive seizure detection by exploiting data from outpatient settings using the Embrace wristband

Conference Paper

Improving convulsive seizure detection by exploiting data from outpatient settings using the Embrace wristband

Abstract

Purpose: Embrace (http://www.empatica.com/product-embrace) is a convulsive seizure detector wristband which relies on traditional accelerometer sensors, and skin conductance sensors, which detect the electrodermal activity triggered by the sympathetic autonomic response during a seizure. We show the effectiveness of using home subjects’ data to dramatically improve the performances of the detector, compare to rely solely on Epilepsy Monitoring Unit (EMU) dataset. Method: Three (3) classifiers, namely EMP0, EMP1 and EMP1+ have been tested. EMP0, based on the original classifier (Poh et al. Epilepsia 2012,53(5),93-7), and EMP1, an improved version of the classifier, were trained on an EMU dataset consisting of 55 generalized convulsive seizures (GCSs) from 69 patients (5,918 hours). EMP1+ is based on EMP1, but was trained on a larger dataset, consisting of home subjects’ data during potentially misleading activities, and of 84 GCSs from 93 patients (6,495 hours) recorded through the Embrace and alert system, which are being evaluated in an IRB-approved clinical trial. The performances were evaluated on a separate testing set with both clinical and home subjects’ data (55 GCSs from 37 patients – 2,210 hours). The recordings not including GCS represents misleading activities. The performances have been evaluated in terms of Sensitivity (Sens) and False Alarm Rate (FAR), i.e. false alarms per 24 hours. Results: EMP1+ outperforms both EMP1 and EMP0. Only EMP1+ classifier can reach Sens=100%, at cost of FAR=5.72. For low (Sens=85%) mid (Sens=90%) and high (Sens=95%) sensitivity, EMP1+ shows respectively FAR=0.85, FAR=1.05 and FAR=2, while EMP1 shows respectively FAR=4.5, FAR=4.85 and FAR=6.1, and EMP0 shows respectively FAR=10.37, FAR=11.35 and FAR=20.48. Conclusion: In this contribution we have demonstrated that having access to home subjects’ data can dramatically improve the performances of a convulsive seizure detector, taking advantage of a more comprehensive pool of human daily activities which can affect the performance of a classifier trained only on EMU data.
Improving convulsive seizure detection by exploiting data
from outpatient settings using the Embrace wristband
Embrace is awearable convulsive seizure (CS) detector
and alert system currently under evaluation in an IRB-
approved clinical trial.The system, shown in Figure 1, relies
on a 3-axes accelerometry (ACM) sensor and askin
conductance sensor, which is able to detect the autonomic
response during aseizure reflected by the electrodermal
activity (EDA).
Data collected in Epilepsy Monitoring Units (EMUs) may not
show as much activity as in real-life, where Embrace is
intended for use.In home settings, people are engaged in
very different activities, such as sports and physical labor,
which can lead to false alarms.
Here we show the advantage of adding data from patients
wearing Embrace at home to the training of the CS detector.
This improves the performance over CS detectors trained
only with data from EMUs.
Three CS detectors (i.e., classifiers), namely EMP0, EMP1and EMP1+, have been trained
and tested as shown in Figure 2. Training data never overlapped with testing data.
EMP0, based on apilot classifier (Poh et al.Epilepsia 2012, 53(5), 93-7), and EMP1, a
revised and improved version of EMP0, were trained on an EMU dataset (EMU_Train)
consisting of data from 69 patients (5,918 hours) including 55 CSs from 22 patients.
EMP1+ is based on EMP1, but was trained on alarger dataset recorded from 92 subjects
(6,495 hours), including EMU_Train and data from 23 home subjects (27 CSs from 4
patients, Home_Train). Home recordings with no seizures were included as examples of
potentially misleading motor activities (e.g., sports, driving, biking).
The performances were evaluated on aseparate testing set from 36 subjects (2,210
hours) including an EMU dataset (EMU_Test) of 5patients (2 CSs from 2patients) and a
home subjects’ dataset (Home_Test) of 31 subjects (53 CSs from 8patients).Ground
truth for EMU CSs was assigned through video-EEG examination, while ground truth for
home subjects’ CSs was derived from patients/caregivers-reported information.
The sensitivity (Sens), i.e. the percentage of recognized seizures, and the False Alarm
Rate (FAR), i.e. the number of false alarms per 24 hours, were computed at different
values of the decision threshold applied to the classifiers’ outputs, for amore complete
and exhaustive analysis.
We have here shown that home data can dramatically improve the performance of a
wearable CS detector with respect to CS detectors trained only on EMU data.
The sensitivity in CS detection reached by the EMP1+ classifier is significantly higher
than EMP0and EMP1classifiers, while maintaining acomfortably low rate of false
alarms.
Francesco Onorati, Ph.D. a, Giulia Regalia, Ph.D. a, Chiara Caborni a, Rosalind Picard, Sc.D. a,b
a. Empatica, Inc., Cambridge, MA, U.S.A
b. MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA, U.S.A
Figure 1. Top:Schematics of Embrace CS detection and alert system, composed of a
wristband that detects an event, transmits an alert to his/her smartphone, which
generates acall via acloud-based service to designated caregivers.Bottom:Detailed
schematic workflow of the CS detector wristband.
Convulsive
seizure
detector
Accelerometers
Skin conductance
sensor
Embrace
Seizure
No seizure
Figure 2. Schematic representation of the training
(top) and testing (bottom) phases of the 3CS detectors
under comparison.
EMP1
classifier
EMP1+
classifier
EMP0
classifier Sens, FAR
EMU_Test +
Home_Test
datasets
TEST
EMU_Train
dataset
EMU_Train +
Home_Train
datasets
EMP1
classifier
EMP1+
classifier
EMP0
classifier
training
training
training
TRAIN
Figure 4.
Performance comparison of the
three seizure detectors on the test
dataset.
Each point of acurve represents
the number of false alarms per day
and the percentage of CSs
correctly recognized at aspecific
value of the decision threshold.
The colored area in the top-left
corner represents the targeted
performances (Sens>90%and
FAR<2false alarms per day).
www.empatica.com/product-embrace
Figure 3shows an example of ACM and EDA signals recorded by Embrace wristband during aCS occurred at home.Figure 4shows the
ROC analysis of the three classifiers for performance comparison.The main results can be summarized as follows:
EMP1performs better than EMP0, but it does not reach the target performances (purple area in Figure 4).
EMP1+ outperforms both EMP0and EMP1, being the only one yielding acceptable Sens and comfortable FAR values (purple area).
EMP1+ shows FAR values 10 times lower than EMP0and 5÷3 times lower than EMP1, when compared at the same Sens.
Only EMP1+ is able to reach Sens=100% (at acost of FAR=5.72).
Figure 3. EDA (top) and ACM (bottom) signals of one
patient recorded during aconvulsive seizure (CS) with
Embrace.The red bars mark the seizure.
EDA (𝛍S)
ACM (g)
5
4
2
0
12 3 4 5 6 7
Time (min)
Purpose
Methods
Results
Conclusions Contacts
12th European Congress on Epileptology, September 11-15, 2016, Prague
On-wrist continous
recordings
Alert generation and
transmission Alert delivery to caregivers
Sens, FAR
Sens, FAR
Francesco Onorati:
fo@empatica.com
Rosalind Picard:
rp@empatica.com
... To train a classification model able to distinguish CS from non-CS events, it was crucial to provide labeled samples to the machine learning algorithm responsible to build the classification model. For this reason, not only previously recorded clinical data, but also previously logged data from real-life activities showing patterns potentially similar to CS (e.g., tooth brushing, hands clapping, hands washing, gesturing, driving or biking on an uneven surface) were used to make a training dataset, as this procedure of showing both good and bad examples showed improved performance on previous preliminary analyses (46). This process was strictly controlled and highly selective to preserve the correct representability and distribution of the data in the training dataset, to avoid mislabeling of data, and most importantly to prevent overlap between training, validation, and testing datasets. ...
Article
Full-text available
Background: Using machine learning to combine wrist accelerometer (ACM) and electrodermal activity (EDA) has been shown effective to detect primarily and secondarily generalized tonic-clonic seizures, here termed as convulsive seizures (CS). A prospective study was conducted for the FDA clearance of an ACM and EDA-based CS-detection device based on a predefined machine learning algorithm. Here we present its performance on pediatric and adult patients in epilepsy monitoring units (EMUs). Methods: Patients diagnosed with epilepsy participated in a prospective multi-center clinical study. Three board-certified neurologists independently labeled CS from video-EEG. The Detection Algorithm was evaluated in terms of Sensitivity and false alarm rate per 24 h-worn (FAR) on all the data and on only periods of rest. Performance were analyzed also applying the Detection Algorithm offline, with a less sensitive but more specific parameters configuration (“Active mode”). Results: Data from 152 patients (429 days) were used for performance evaluation (85 pediatric aged 6–20 years, and 67 adult aged 21–63 years). Thirty-six patients (18 pediatric) experienced a total of 66 CS (35 pediatric). The Sensitivity (corrected for clustered data) was 0.92, with a 95% confidence interval (CI) of [0.85-1.00] for the pediatric population, not significantly different (p > 0.05) from the adult population's Sensitivity (0.94, CI: [0.89–1.00]). The FAR on the pediatric population was 1.26 (CI: [0.87–1.73]), higher (p < 0.001) than in the adult population (0.57, CI: [0.36–0.81]). Using the Active mode, the FAR decreased by 68% while reducing Sensitivity to 0.95 across the population. During rest periods, the FAR's were 0 for all patients, lower than during activity periods (p < 0.001). Conclusions: Performance complies with FDA's requirements of a lower bound of CI for Sensitivity higher than 0.7 and of a FAR lower than 2, for both age groups. The pediatric FAR was higher than the adult FAR, likely due to higher pediatric activity. The high Sensitivity and precision (having no false alarms) during sleep might help mitigate SUDEP risk by summoning caregiver intervention. The Active mode may be advantageous for some patients, reducing the impact of the FAR on daily life. Future work will examine the performance and usability outside of EMUs.
... SDD technology continues to advance, with the FDA issuing 510(k) clearances for two wearable SDDs [8,9]. Recent research has also resulted in the publication of several validation studies that assess device detection performance and additional reporting capabilities [10,11]. ...
Article
Objective This study characterizes the current capabilities of seizure detection device (SDD) technology and evaluates the fitness of these devices for use in anti-seizure medication (ASM) clinical trials. Methods Through a systematic literature review, 36 wireless SDDs featured in published device validation studies were identified. Each device’s seizure detection capabilities that addressed ASM clinical trial primary endpoint measurement needs were cataloged. Results The two most common types of seizures targeted by ASMs in clinical trials are generalized tonic-clonic (GTC) seizures and focal with impaired awareness (FIA) seizures. The Brain Sentinel SPEAC achieved the highest performance for the detection of GTC seizures (F1-score = 0.95). A non-commercial wireless EEG device achieved the highest performance for the detection of FIA seizures (F1-score = 0.88). Discussion A preliminary assessment of device capabilities for measuring selected ASM clinical trial secondary endpoints was performed. The need to address key limitations in validation studies is highlighted in order to support future assessments of SDD fitness for ASM clinical trial use. In tandem, a stepwise framework to streamline device testing is put forth. These suggestions provide a starting point for establishing SDD reporting requirements before device integration into ASM clinical trials.
... Taking another Intelligent and Interactive Companion Combined With Wearable Technology and Re-creatable Environment to Avoid Anxiety H. Senaratne 2 step ahead, wearable technology has come into the play in order to monitor the stress level of a person. Projects like Spire [7] and Embrace [8] monitor the stress level of an individual by measuring different attributes such as respiration patterns, heart rate and skin conductance and also suggest the activities they can perform when stressed moments are monitored. WeaRelaxAble is a design proposed to provide various feedback modalities, such as vibration, ambient light, acoustic stimuli and heat in order to reduce the user's stress level, which includes with a shirt to wear by the user and a wrist worn device to trigger the stimuli [9]. ...
Conference Paper
With the modernization of the world, people have led into more stressful lifestyles. Due to long lasting stress, many individuals experience anxiety conditions and this problem has become a common problem in every society of the world. To solve this issue, technology has come into play with suggestions of different types of solutions. In this paper we have come up with a proposal that can become a future work in the human computer interaction field, which suggests an intelligent and interactive companion supported with wearable technology to monitor the stress level of the user and a controllable environment with the potential of recreating moments, in order to avoid anxiety. Although there are existing wearable devices that help to monitor stress levels and some initial level research has been done to come up with companions to reduce stress, integration of a wearable device along with an interactive companion will be a newer level of research. Also identifying and capturing environmental factors which affects one’s stress level and changing environmental factors to reduce the stress level, is a less studied area. Therefore the suggested future work exposes a research area which has the capability of expanding a lot in the future.
Conference Paper
Full-text available
RATIONALE: Empatica (www.empatica.com) is working on the development of an automated comfortably wearable convulsive seizure (CS) detection system relying on accelerometer (ACC) and electrodermal activity (EDA) data (Epilepsia 2012, 53, 93-7). Using machine learning algorithms trained on generalized tonic-clonic seizures (GTCS) gathered inside Epilepsy Monitoring Units (EMUs), the system has achieved sensitivity (Se) of 92-100% and false alarm (FA) rates ranging from 0.48 to 2.02 false alarms per day (Regalia et al. 2015, American Epilepsy Society Annual Meeting; Onorati et al. 2016, Epilepsy Pipeline Conference, Caborni et al. 2016, Partners Against Mortality in Epilepsy). However, EMU settings do not mimic the real-life environment where the system is intended for use. In real-life settings people are engaged in very different physical activities, such as sports and physical labor, which may result in higher FA rates. Moreover, the dynamic and semiology of CSs occurring outside EMUs might be different, influencing their detection. In this work we present a case study from a real-life setting. METHODS: Embrace is a wrist-worn device and smartphone-based alert system which analyses 3-axis acceleration and EDA data from the patient and provides an alert to designated caregivers when an unusual event is detected. The Embrace and alert system are being evaluated in an IRB-approved clinical trial. This case study is of a patient with Dravet Syndrome (14 y.) enrolled in the trial of Embrace in the outpatient setting. No data from this patient was used in training the system. In order to evaluate performance, the patient’s caregiver was asked to meticulously annotate the occurrence of each CS and any activity that generated an alert. The number of FA’s was obtained by subtracting the number of correctly recognized CSs from the total alerts fired by the device. The Se was the percentage of CSs that automatically triggered an alert. RESULTS: Over a period of 113 days, the patient wore the device for 82 days (i.e., 1973 hours, average hours per day: 17.2). The system detected 22 out of 24 CSs (Se=92%). The 2 missed seizures were characterized by a mild motor component and brief duration ( < 50 sec). Figure 1 depicts the distribution of the patient’s seizures according to the hour of the day. The total number of FA was 39, for a FA rate of 0.48 per day worn. FAs were generated by activities such as hands clapping/shaking, car transport and dancing. CONCLUSIONS: In this work, we have reported the performance of an unobtrusive CS detector used by a patient for a period of more than 3 months in a real-life setting, where none of the patient’s data had been used in training the system. The performance, both Se and FA rates, were in the same range as those for data gathered in best-case clinical settings. In ongoing research with more patients, we are seeing similar results. At AES, and in future publications, we will present more detailed evaluations with other patients and healthy subjects engaged in diverse real-life activities.
ResearchGate has not been able to resolve any references for this publication.