An EEG Experimental Study Evaluating the Performance of Texas Instruments ADS1299

Abstract

Texas Instruments ADS1299 is an attractive choice for low cost electroencephalography (EEG) devices owing to its low power consumption and low input referred noise. To date, there have been no rigorous evaluations of its performance. In this EEG experimental study we evaluated the performance of the ADS1299 against a high quality laboratory-based system. Two self-paced lower limb motor tasks were performed by 22 healthy participants. Recorded power across delta, theta, alpha, and beta EEG bands, the power ratio across the motor tasks, pre-movement noise, and signal-to-noise ratio were obtained for evaluation. The amplitude and time of the negative peak in the movement-related cortical potentials (MRCPs) extracted from the EEG data were also obtained. Using linear mixed models, no statistically significant differences (p > 0.05) were found in any of these measures across the two systems. These findings were further supported by evaluation of cosine similarity, waveform differences, and topographic maps. There were statistically significant differences in MRCPs across the motor tasks in both systems. We conclude that the performance of the ADS1299 in combination with wet Ag/AgCl electrodes is analogous to that of a laboratory-based system in a low frequency (<40 Hz) EEG recording.

Full text

sensors
Article
An EEG Experimental Study Evaluating the
Performance of Texas Instruments ADS1299
Usman Rashid 1,* , Imran Khan Niazi 1,2,3 , Nada Signal 1and Denise Taylor 1
1Health & Rehabilitation Research Institute, Auckland University of Technology, Auckland 0627,
New Zealand; imran.niazi@aut.ac.nz (I.K.N.); nada.signal@aut.ac.nz (N.S.); denise.taylor@aut.ac.nz (D.T.)
2Centre for Chiropractic Research, New Zealand College of Chiropractic, Auckland 1060, New Zealand
3SMI, Department of Health Science and Technology, Aalborg University, Aalborg 9220, Denmark
*Correspondence: urashid@aut.ac.nz
Received: 9 October 2018; Accepted: 30 October 2018; Published: 1 November 2018


Abstract:
Texas Instruments ADS1299 is an attractive choice for low cost electroencephalography
(EEG) devices owing to its low power consumption and low input referred noise. To date, there
have been no rigorous evaluations of its performance. In this EEG experimental study we evaluated
the performance of the ADS1299 against a high quality laboratory-based system. Two self-paced
lower limb motor tasks were performed by 22 healthy participants. Recorded power across delta,
theta, alpha, and beta EEG bands, the power ratio across the motor tasks, pre-movement noise, and
signal-to-noise ratio were obtained for evaluation. The amplitude and time of the negative peak in
the movement-related cortical potentials (MRCPs) extracted from the EEG data were also obtained.
Using linear mixed models, no statistically significant differences (p> 0.05) were found in any of
these measures across the two systems. These findings were further supported by evaluation of
cosine similarity, waveform differences, and topographic maps. There were statistically significant
differences in MRCPs across the motor tasks in both systems. We conclude that the performance of
the ADS1299 in combination with wet Ag/AgCl electrodes is analogous to that of a laboratory-based
system in a low frequency (<40 Hz) EEG recording.
Keywords:
ADS1299; OpenBCI Cyton V3-32; electroencephalography (EEG); movement-related
cortical potential (MRCP); brain computer interface (BCI); NuAmps
1. Introduction
Texas Instruments ADS1299 is a system on chip (SOC) specifically designed for biopotential
applications including electroencephalography (EEG) and electrocardiography (ECG). It has attractive
electrical characteristics such as low input referred noise (1
µ
V
pp
), low power consumption
(5 mW/channel), test signals for impedance measurement, and an SPI
™
compatible interface [
1
].
In recent years, it has attracted considerable attention from the biomedical research community,
both for clinical research and the development of mobile brain computer interfaces (BCIs).
ADS1299 has been used in multiple studies to evaluate hardware and software for EEG.
These include testing 3D printed electrodes [
2
], ultra high impedance active electrodes [
3
], finger-based
dry electrodes [
4
], and artefact rejection algorithms [
5
,
6
]. It has also been used in EEG experimental
research to study neurophysiology in both humans [
7
–
10
] and mice [
11
]. Importantly, these studies are
making decisions about the relative merits of these technologies and experimental outcomes based on
parameters of the EEG recorded with the ADS1299.
With an increased interest in mobile brain computer interfaces [
12
–
14
], engineers and researchers
have sought front-ends which are low cost, have a small form factor, and possess good electrical
characteristics [
15
]. Consequently, in the past two years alone, they have appeared in six novel BCIs,
Sensors 2018,18, 3721; doi:10.3390/s18113721 www.mdpi.com/journal/sensors

Sensors 2018,18, 3721 2 of 18
namely, an interactive care system for aged patients with dementia [
16
], a modular hybrid BCI based
on EEG and near infra-red spectroscopy (NIRS) [
17
], a plug and play BCI for active and assisted
living control [
18
], a hybrid BCI combining P300 and auditory steady state response (ASSR) [
19
],
an embedded BCI for classification of event related synchronisation/desynchronisation [
20
], and a
study of stimuli design for a BCI based on steady state evoked potentials (SSVEP) [
21
]. However,
these BCIs have been tested in only a small number of participants and evaluations of the signal quality
have relied on comparisons to past literature.
The breadth of research using the ADS1299 illustrates its appeal for both clinical research and
end-user devices. None of these studies has evaluated the EEG recording performance of the ADS1299
in comparison with a high quality system in absence of other variables such as novel electrodes,
an algorithm, or a BCI paradigm. Researchers have emphasised the need for objective evaluation of
EEG signals from devices intended to be used in BCI applications [
22
–
24
]. Thus, there is a pressing
need for an an independent evaluation of the ADS1299 with a robust experimental design. The aim
of this research is to evaluate its performance in EEG against a high quality laboratory-based system
while controlling for participants, paradigm, tasks performed, electrodes used, and data processing
methods. This research also aims to evaluate the performance of the ADS1299 in both single joint
(dorsiflexion while sitting) and multi-joint (step on/off while standing) motor tasks.
2. Materials and Methods
This within-participants experimental study was conducted at Auckland University of Technology,
New Zealand. Ethical approval for the study (17/CEN/133) was obtained from Central Health and
Disability Ethics Committee (HDEC), New Zealand, in accordance with the Declaration of Helsinki.
2.1. Participants
Twenty-two healthy participants (average age: 36
±
6 years, 10 female) were recruited through
professional networks and local advertising. Participants were excluded if they had a history of
any neurological disorders or epilepsy. All the participants signed a written consent form before
data collection.
2.2. EEG Systems
2.2.1. Prototype
For evaluation of the ADS1299, a prototype system was built using OpenBCI (OpenBCI, New York,
NY, USA) V3-32 board along with a V3 Daisy module for supporting up to 16 channels, a 4N25
optocoupler, and required connectors. The firmware of the OpenBCI board was modified from version
3.0.0 (refer to supplementary files). The data were saved on the onboard SD card. The sampling
rate was set at 250 Hz and programmable gain (PGA) at 24, as these settings result in minimum
peak-to-peak input referred noise [1].
2.2.2. Gold Standard
NuAmps (Compumedics Neuroscan, Dresden, Germany) was used as the gold standard system
(GS) [
25
–
29
]. A search on Google Scholar for the keywords “Compumedics Neuroscan NuAmps”
returned 395 results. It was connected to a computer via a USB cable and data were recorded with
the Acquire software (Compumedics Neuroscan, Dresden, Germany). As per standard data collection
protocols, the sampling rate was set at 500 Hz [25–29].
2.2.3. Electrodes
With both the systems, a 32 channel Quick-Cap with Ag/AgCl wet electrodes (Compumedics
Neuroscan, Dresden, Germany) was used for EEG, and disposable BlueSensor N (Ambu
®
, Bayan

Sensors 2018,18, 3721 3 of 18
Lepas, Malaysia) electrodes were used for surface electromyography (sEMG). A 37 pin D-subminiature
connector was used to connect the prototype to the Quick-Cap.
2.3. Experiment Protocol
For each of the 22 participants, data were collected over two sessions on consecutive days. In each
session, participants performed two motor tasks using a single system. The order of the systems and the
order of the motor tasks was randomised across the participants. Thus, all the participants performed
both the tasks using both the systems, but the order in which they performed these tasks and the choice
of the system used on a given day was random. This pair-wise matching of the participants across
the two systems and the two tasks allowed for the paired statistical analysis. This within-participant
protocol is in agreement with the previous research evaluating EEG hardware [22,23].
In each session, the participants executed 50 right foot ballistic dorsiflexions while seated and
50 repetitions of right foot step on and off a step-stool (approximately 23 cm high) placed at a
comfortable distance while standing. Participants were asked to place their right foot on the step-stool
and immediately bring it back to the ground. They executed the tasks at their own pace while pausing
for at least 5 s between each repetition. The order of the tasks was chosen at random, and data were
recorded separately for each task.
EEG data were collected from 14 International 10–20 system locations (Fp1, F3, F4, FC3, FCz, FC4,
C3, Cz, C4, CP3, CPz, CP4, P3, and P4) using either the gold standard or the prototype depending
on the randomisation schedule. A single reference electrode from Quick-Cap was used, which was
located on the right mastoid. Electrodes were prepared using Quick-Gel (Compumedics Neuroscan,
Dresden, Germany). For sEMG, two electrodes were placed on the right Tibialis Anterior (TA) muscle.
Preparation included shaving, exfoliating with the Nuprep Gel (Weaver and Company, Aurora, CO,
USA), and cleansing with disposable alcohol swabs. Acquire software (Compumedics Neuroscan,
Dresden, Germany) was used in combination with NuAmps to monitor impedance for both EEG and
sEMG, and was accepted when below 10 k
Ω
. During data collection, the systems were placed on a
desk next to the participant.
2.4. Data Processing
The data were processed on MATLAB 2017b (MathWorks, Inc., Natick, MA, USA) using a
combination of custom code (refer to supplementary files) and EEGLAB (version 14.1.1) functions [
30
].
Movement onsets from rectified sEMG data were detected using an extended version of the double
thresholding algorithm [
31
]. These onsets were then visually checked and adjusted. Using 2nd order,
zero-phase, Butterworth filters, sEMG data were filtered with a high pass cut-off at 10 Hz, and a low
pass cut-off at 200 Hz and at 100 Hz for the gold standard and the prototype, respectively. As sEMG
was used only for identifying the movement onsets, these cut-offs were considered adequate [32,33].
EEG Processing
EEG channels were filtered with 2nd order, zero phase, Butterworth filters with a highpass filter
cut-off at 0.05 Hz, a low pass filter cut-off at 40 Hz, and a notch filter from 49 to 51 Hz. These filtered EEG
channels were visually inspected along with their frequency spectrum and histograms. The channels
which were missing or had very large transients were removed and interpolated with EEGLAB
pop_interp function using the spherical interpolation method. The data were then down-sampled to
125 Hz, and epochs were derived by including data from 3 s before and after the sEMG onset.
Then, epochs with large or very fast transients were removed and independent component
analysis (ICA) was performed with EEGLAB pop_runica function using the runica algorithm [
34
].
Components corresponding to eye blinks, or limited to only one electrode and a few epochs,
were removed. The remaining components were remixed (back-projected to sensor space), and
epochs with an amplitude above 125
µ
V
pp
across any channel were removed to obtain clean EEG
epochs. Movement-related cortical potential (MRCP) epochs were obtained from these cleaned EEG

Sensors 2018,18, 3721 4 of 18
epochs by applying a small spatial filter across FC3, FCz, FC4, C3, Cz, C4, CP3, CPz, and CP4 channels
with the center at Cz [
35
], followed by a 2nd order, zero phase, Butterworth filter with a low pass
cut-off at 5 Hz. By taking the mean across the epochs, the averaged MRCP was obtained.
2.5. Performance Measures
In order to compare the two systems, the following performance measures were obtained from
the EEG epochs and the averaged MRCP for each participant.
2.5.1. EEG Specific Measures
a.
Power across EEG Bands: Power across four EEG bands (delta [0.05–3 Hz], theta [3–8 Hz],
alpha [8–12 Hz]
, and beta [12–38 Hz]) was obtained separately from each epoch for all the
channels using MATLAB bandpower function. The obtained power was converted to decibels
(dB) and then averaged across epochs and channels for each band [22].
b.
Pre-Movement Noise (PMN): The EEG activity from 2–3 s before the sEMG onset was regarded
as baseline [
36
]. The root mean square (RMS) value of the baseline was calculated separately
from each epoch for all the channels and then averaged across channels and epochs to obtain
pre-movement noise (PMN) [37].
2.5.2. MRCP Specific Measures
a.
Signal-to-Noise Ratio (SNR): The signal-to-noise ratio was defined as the ratio of peak negative
amplitude to the RMS value in the baseline segment of the averaged MRCP, expressed in decibels.
b.
Amplitude and Time of Negative Peak (PN, PNT): The amplitude of the negative peak in the
1 s
before and after the sEMG onset was obtained from the averaged MRCP using a local peak
algorithm [
38
]. The time of the peak negative amplitude was expressed in milliseconds, where a
negative value represents occurrence before, and a positive value after, the sEMG onset. The peak
negative value is one of the most important features of the MRCP as it has been widely studied
in relation with rehabilitation and motor learning [39,40].
2.6. Statistical Analysis
Statistical analysis was performed in R (R Foundation for Statistical Computing) version 3.5.0
(refer to supplementary files). Means are given with the standard deviations. Based on an a priori
statistical analysis, the performance of the prototype was assessed against the gold standard for
power across EEG bands, power ratio, pre-movement noise, signal-to-noise ratio, and the amplitude
and time of the negative peak using linear mixed models or generalised linear mixed models as
appropriate [
41
]. lme4 package version 1.1-17 was used for fitting the models [
42
]. The detailed setup
of the models is given in the corresponding result sections. Analysis of deviance tables for the models
were obtained using the Anova function from car package version 3.0-0 [
43
]. Significance level was
set at 0.05. For main effects and interactions, Type II Wald chi-square tests are reported. In case of
significant interactions, pair-wise comparisons were performed with Tukey’s HSD (honest significant
differences) method using the lsmeans function from the lsmeans package version 2.27-62 [
44
]. For the
pair-wise comparisons, Cohen’s
d
effect size using pooled variance was calculated. It was interpreted
as a small (0.20), medium (0.5), or large (0.8) effect [
45
]. Cosine similarity was also assessed for the
averaged MRCPs across the two systems. The MRCPs from the two systems were treated as vectors
and their cosine similarity was defined as follows [46].
r=
u.v
kuk × kvk(1)
where
u
,
v
represent the two MRCPs as vectors. ‘
.
’ represents the dot product between the two vectors,
and
k
.
k
represents the L
2
norm of the vector. Two additional analyses which were not part of the a

Sensors 2018,18, 3721 5 of 18
priori analysis plan were also performed. First, grand average MRCPs for the gold standard and the
prototype were obtained in dorsiflexion and step on/off from the averaged MRCPs of the participants.
These grand average MRCPs were plotted along with their differences and 95% confidence intervals.
Differences were evaluated by applying sample wise repeated measure t-tests and interpreted based on
Bonferroni corrected p-values. Second, interpolated topographic maps were also obtained at different
latencies by task and system wise pooling of all the cleaned EEG epochs. The pop_topoplot function
from EEGLAB was used for this purpose. Differences were evaluated by obtaining the maps from
absolute difference of the channels.
3. Results
3.1. Data Loss and Artefacts
In case of the gold standard system, there was no loss of data. For the prototype, the data
from dorsiflexion of one participant were lost as the sEMG channels were recorded for only the first
repetition. The EEG data from this repetition were rejected due to the presence of large transients.
The prototype’s data had occasional single-sample very-large-amplitude (>1000
µ
V) artefacts in some
channels as shown in Figure 1. The incidence of these artefacts was approximately 1 in 10,000 samples.
It was removed using an automatic detection algorithm and replaced with the average of the fifth
sample from its left and right.
0 2 4 6 8 10 12
Sample (n) 104
-1200
-1000
-800
-600
-400
-200
0
200
Amplitude (uV)
Raw channel data
Impulse artefact
Replaced value
Figure 1. An example of the artefact detected in the prototype data.
3.2. Rejection of Channels, ICA Components, and Epochs
For the gold standard system, on average 0.5
±
0.8 and 0.7
±
1.5 channels were interpolated in
dorsiflexion and step on/off respectively, across the participants. For the prototype, 0.3
±
0.6 and
0.4 ±0.7
channels were interpolated. During the ICA analysis, components corresponding to eye
blinks, or limited to only one electrode and a few epochs, were removed. For the gold standard
3.0 ±1.9
and
3.8 ±2.1
components were rejected in dorsiflexion and step on/off, respectively. For the
prototype,
3.4 ±2.2
and
3.3 ±1.8
components were removed. The number of interpolated channels
and rejected components appear to be similar across systems and higher in step on/off compared
to dorsiflexion.
Two sets of 50 EEG epochs corresponding to the two systems were obtained for each task from
the sEMG onsets. The epoch rejection rate was calculated as the ratio of the number of epochs rejected
manually and with thresholding to number of sEMG onsets used for creating epochs. This ratio was

Sensors 2018,18, 3721 6 of 18
expressed as a percentage. The mean rejection rate at 125
µ
V
pp
ranged from 4.9 to 7.6%. However,
when 75 µVpp
threshold was applied, it ranged from 31.9 to 52.5% across tasks and systems, refer to
Table 1. Whilst the standard deviation of the dorsiflexion rejection rate appears higher in the prototype,
this can be explained by the rejection of 1 out of 1 epoch for the participant whose sEMG data were
lost as explained earlier in Section 3.1.
Table 1.
Means and standard deviations for percentage epoch rejection rates at 75
µ
V
pp
and 125
µ
V
pp
.
Task System Rejection at 75 µVpp (%) Rejection at 125 µVpp (%)
Dorsiflexion GS 31.9 ±26.4 6.1 ±7.0
Proto 34.7 ±31.7 7.6 ±20.9
Step on/off GS 52.5 ±35.6 5.7 ±6.2
Proto 43.6 ±34.1 4.9 ±4.0
3.3. EEG Specific Measures
3.3.1. Power Across EEG Bands
Separate linear mixed models were set up to compare EEG power in dorsiflexion and step on/off,
respectively. Systems, bands, and an interaction term for systems and bands were entered as fixed
effects. For participants, a random intercept term was also entered. The setup of the linear mixed
model is given in R formula syntax as follows [42].
Power ~System + EEGBand + System *EEGBand + ( 1| P a r t i c i p a n t )
In dorsiflexion, there was no interaction between the systems and EEG bands
(χ2[3] = 1.82, p= 0.61)
. Moreover, there was no significant difference across systems
(χ2[1] = 0.01, p= 0.91).
There was a significant difference in power across bands (
χ2
[3] = 554.42,
p< 0.001). Similar results were obtained in step on/off. There was no interaction between the systems
and EEG bands
(χ2[3] = 0.57, p= 0.90)
. There was no significant difference across systems (
χ2
[1] = 0.13,
p= 0.72). There was a significant difference across the bands (
χ2
[3] = 508.33, p< 0.001) (see Figure 2A,B
and Table 2). These results indicate that the prototype system is comparable to the gold standard
system with respect to power.
Table 2.
Means and standard deviation for power values in decibels (dB) across four EEG bands in
dorsiflexion and step on/off.
Task System Delta (dB) Theta (dB) Alpha (dB) Beta (dB)
Dorsiflexion GS 17.2 ±1.4 9.4 ±1.9 8.4 ±3.5 10.5 ±2.6
Proto 17.9 ±2.2 9.3 ±2.4 8.5 ±3.3 10.0 ±2.8
Step on/off GS 18.0 ±1.8 9.9 ±1.9 9.3 ±4.4 11.3 ±2.5
Proto 18.5 ±2.0 9.9 ±2.1 9.4 ±3.4 11.3 ±2.5
In order to assess the sensitivity of the systems to detect differences in power across tasks, a
participant wise ratio of power values from step on/off to dorsiflexion (S/D) was obtained, as shown
in Figure 2C. For the delta band, the power ratios for the gold standard and the prototype were
1.05 ±0.12
and
1.04 ±0.13
, respectively. The power ratios for the prototype in theta, alpha, and beta
bands were 1.1
±
0.27, 1.2
±
0.43, and 1.18
±
0.33, respectively. For the gold standard, the ratios for
the theta, alpha, and beta bands were 1.08
±
0.19, 1.12
±
0.27, and 1.1
±
0.17, respectively. A linear
mixed model with systems, bands, and an interaction term for systems and bands as fixed effects
was set up. A random intercept term for the participants was also entered. Significant interaction
between the bands and the systems on the power ratio was not detected (
χ2
[3] = 1.51, p= 0.68). There
were no significant differences across bands (
χ2
[3] = 6.8, p= 0.08) or systems (
χ2
[1] = 1.94, p= 0.16).

Sensors 2018,18, 3721 7 of 18
These results suggest that both systems have similar sensitivity to power differences in EEG bands
across different motor tasks.
Dorsiflexion
***
Delta Theta Alpha Beta
0
10
20
30
Power (dB)
GS
Proto
Step on/off
***
Delta Theta Alpha Beta
0
10
20
30
Power (dB)
GS
Proto
Delta Theta Alpha Beta
0
1
1.2
1.4
1.6
1.8
2
Ratio S/D
GS
Proto
A. B.
C.
Figure 2.
Power recorded across four EEG bands in (
A
) dorsiflexion and (
B
) step on/off . (
C
) Participant
wise ratio of power values from step on/off to dorsiflexion (S/D). ‘***’ represents p-value less than 0.001.
3.3.2. Pre-Movement Noise
Pre-movement noise in dorsiflexion was 8.82
±
1.57
µ
Vrms and 8.85
±
2.00
µ
Vrms for the
gold standard and the prototype respectively. In step on/off the values for the two systems were
9.87 ±1.91 µVrms
and 10.11
±
2.54
µ
Vrms. To evaluate differences across systems and tasks in the
pre-movement noise, a generalised linear mixed model with the Gamma family and log link was set
up as the data had skew with a long tail. Systems, tasks and an interaction term for systems and tasks
were entered as fixed effects. A random intercept term for the participants was also entered. The setup
of the generalised linear mixed model is given in R formula syntax as follows [42].
No ise ~System + Ta sk + System *T as k + ( 1 | P a r t i c i p a n t )
There was no significant interaction between systems and tasks on pre-movement noise
(χ2[1] = 0.09, p= 0.77).
There was no significant difference across the systems (
χ2
[1] = 0.05, p= 0.83).
A significant difference was detected across the tasks (
χ2
[1] = 14.12, p< 0.001). These results suggest
that the prototype system’s susceptibility to noise is similar to the gold standard system. However,
both systems have higher noise in step on/off compared to dorsiflexion, refer to Figure 3A.

Sensors 2018,18, 3721 8 of 18
***
Dorsiflexion Step on/off
0
5
10
15
PMN (uVrms)
***
Dorsiflexion Step on/off
0
5
10
15
SNR (dB)
GS
Proto
A. B.
Figure 3.
Means and standard deviations for (
A
) pre-movement noise and (
B
) signal-to-noise ratio.
‘***’ represents p-value less than 0.001.
3.4. MRCP Specific Measures
3.4.1. Signal-to-Noise Ratio
Signal-to-noise ratio in dorsiflexion was 5.56 ±2.06 dB and 6.13 ±2.67 dB for the gold standard
and the prototype respectively. In step on/off the values for the two systems were
2.99 ±2.85 dB
and
2.09 ±2.17 dB
. A linear mixed model with systems, tasks and an interaction term for systems
and tasks as fixed effects was setup. A random intercept term for the participants was also entered.
Significant interaction between the systems and the tasks was not detected (
χ2
[1] = 2.32, p= 0.13).
No significant differences were detected across the systems (
χ2
[1] = 0.111, p= 0.74). However,
a significant difference was detected across the tasks (
χ2
[1] = 44.75, p< 0.001). These results indicate
that the signal-to-noise ratio of the prototype is comparable to that of the gold standard, refer to
Figure 3B.
3.4.2. Amplitude and Time of the Negative Peak
Peak negative value in dorsiflexion for the gold standard and the prototype system was
−4.44 ±2.17 µV
and
−4.29 ±2.00 µV
respectively. In step on/off the PN amplitude was
−2.36 ±1.34 µV
and
−2.75 ±1.50 µV.
A linear mixed model with systems, tasks and an interaction
term for systems and tasks as fixed effects was setup. A random intercept term for the participants was
also entered. Interaction between the systems and the tasks on the PN amplitude was not significant
(
χ2
[1] = 1.04, p= 0.31). There were no significant differences across the systems (
χ2
[1] = 0.23, p= 0.63).
A significant difference across the tasks was detected (χ2[1] = 47.61, p< 0.001).
The time of the peak negative amplitude in dorsiflexion was
261.09 ±139.32 ms
and
256.00 ±181.73 ms
for the gold standard and the prototype system respectively. In case of step
on/off, the time for the systems was
284 ±340.24 ms
and
397.82 ±344.05 ms
. A linear mixed model
with systems, tasks and an interaction term for systems and tasks as fixed effects was setup. A random
intercept term for the participants was also entered. Interaction between the systems and the tasks
on the time of the peak negative amplitude was not significant (
χ2
[1] = 1.23, p= 0.27). There were no
significant differences across the systems (χ2[1] = 1.11, p= 0.29) and the tasks (χ2[1] = 2.31, p= 0.13).
The systems were comparable with respect to the time and the amplitude of the negative peak in
the MRCP. The amplitude of the negative peak differed significantly between the motor tasks, refer
to Figure 4.

Sensors 2018,18, 3721 9 of 18
***
Dorsiflexion Step on/off
0
-1
-2
-3
-4
-5
-6
-7
-8
PN amplitude (uV)
Dorsiflexion Step on/off
0
100
200
300
400
500
600
700
800
PN time (ms)
GS
Proto
A. B.
Figure 4.
Means and standard deviations for (
A
) the peak negative value and (
B
) its time for the two
systems across the tasks. ‘***’ represents p-value less than 0.001.
3.5. Cosine Similarity
The mean similarity in dorsiflexion across the two systems was 0.84
±
0.14 and 0.75
±
0.28 in step
on/off. Averaged MRCPs from the two systems along with cosine similarity are given in Appendix A.
The results indicate a strong cosine similarity between the MRCP signals recorded by the two systems,
suggesting that the prototype is comparable to the gold standard.
3.6. Grand Average of Participant MRCPs
Grand averages and difference waveforms with 95% confidence intervals obtained from the
averaged MRCPs of the individual participants are shown in Figures 5and 6. The grand average
MRCPs were similar for the two systems in both the tasks. The confidence intervals were uniform,
overlapping, and their size was smaller than
±
1
µ
V. There were no significant differences across the
systems. These results indicate agreement between the averaged MRCPs recorded by the two systems.
−5
−4
−3
−2
−1
0
1
2
−2.0 −1.5 −1.0 −0.5 0.0 0.5 1.0
Time (s)
Amplitude (µV)
Difference
GS
Proto
Figure 5.
Grand averages and difference (GS-Proto) along with 95% confidence intervals for the
averaged MRCPs (n= 21). Time at 0 s corresponds to the movement onset.

Sensors 2018,18, 3721 10 of 18
−3
−2
−1
0
1
2
3
−2.0 −1.5 −1.0 −0.5 0.0 0.5 1.0
Time (s)
Amplitude (µV)
Difference
GS
Proto
Figure 6.
Grand averages and difference (GS-Proto) along with 95% confidence intervals for the
averaged MRCPs (n= 22). Time at 0 s corresponds to the movement onset.
3.7. Topographic Maps
Interpolated topographic maps obtained from cleaned EEG epochs of the 14 EEG channels are
shown in Figures 7and 8for dorsiflexion and step on/off, respectively. These maps suggest that
in both tasks, the spatial distribution of cortical activity was similar, and the differences across the
systems are negligible. At movement onset, cortical activations are centered around CPz and Cz in
both systems. These results also indicate that both systems record similar EEGs.
(a)
(b)
(c)
Figure 7.
Interpolated topographic maps obtained at different latencies with respect to the movement
onset form cleaned EEG epochs (n= 21); 977 and 1007 epochs were used for the gold standard and the
prototype, respectively. Time at 0 ms corresponds to the movement onset. (
a
) GS; (
b
) Proto; (
c
) Absolute
difference of channels from (a) and (b).

Sensors 2018,18, 3721 11 of 18
(a)
(b)
(c)
Figure 8.
Interpolated topographic maps obtained at different latencies with respect to the movement
onset form cleaned EEG epochs (n= 22); 1022 and 1028 epochs were used for the gold standard and the
prototype, respectively. Time at 0 ms corresponds to the movement onset. (
a
) GS; (
b
) Proto; (
c
) Absolute
difference of channels from (a) and (b).
4. Discussion
This experimental study has rigorously evaluated the EEG recording performance of ADS1299 in
comparison to a high quality laboratory-based system whilst controlling for participants, paradigm,
tasks performed, electrodes used, and data processing methods in a large sample of healthy participants.
The main findings of this research are that the ADS1299 is comparable with respect to power across
EEG bands, power ratio, pre-movement noise of the EEG, and the signal-to-noise ratio, the amplitude
and time of the negative peak, and cosine similarity of the MRCPs. The validity of these findings
are supported by the large sample size and the robust statistical analysis, which accounted for
between-participant variance. These results are discussed in detail below.
We computed the average power in delta, theta, alpha, and beta bands from all the included
epochs of all the recorded channels. Thus, the obtained value included both the evoked and the induced
power. Within both motor tasks, there were no significant differences between the laboratory-based
system and the ADS1299. The sensitivity to power changes across different tasks was evaluated with
the ratio of step on/off power to dorsiflexion. There were no differences in the power ratio across the
systems or the bands. Therefore, in EEG studies where spectral features during motor tasks are of
interest [47–49], ADS1299 can be used with confidence.
Another important measure of EEG device quality is the level of noise present in the baseline.
Similar to past studies [
22
,
37
], this was quantified by the root mean square value in the baseline
EEG from epochs of all channels. There were no differences across the two systems. The noise was
higher in the case of step on/off compared to dorsiflexion. The most likely sources of this noise are
cable sway [
50
], head movements [
51
], or physiological differences. In this research, the experimental
protocol and the requirement to standaradise the setup across the systems necessitated that the
ADS1299 was placed on a desk. However, in mobile BCI applications, the effects of head movements
and cable sway may be mitigated by the use of an inertial measurement unit and by mounting it closer
to the electrodes [
52
]. Comparable noise levels in both systems is a strong indicator of the quality of
EEG recorded by the ADS1299.

Sensors 2018,18, 3721 12 of 18
Two notable differences between the systems were revealed during data pre-processing. First, the
prototype data exhibited a single sample, very large value artefact. This problem most likely originated
from the way data were stored on the SD card using the OpenBCI board, as the error was also found
in the raw data file before conversion from hexadecimal values. The artefact could not be removed
with a low pass filter as this resulted in large transients. However, the artefact sample could easily
be substituted with an averaged value. The second difference between the systems was that, in one
prototype file, sEMG data were missing. The source of this problem is most likely that the Daisy board
was disconnected from the main OpenBCI board and failed to reconnect during recording. In future
studies, this problem can be avoided by ensuring a more robust connection between the two boards.
A comparable number of interpolated channels, rejected ICA components, and epoch rejection
rates indicate that the ADS1299 records EEG reliably. We explored epoch rejection rates at traditional
75 and 125 µVpp
thresholds. At 75
µ
Vpp, both systems had rejection rates of 40–50% during step
on/off, which is similar to those reported by Oliveira et al. for the Biosemi system and the Cognionics
wet system during a walking task [
22
]. At 125
µ
Vpp, the ADS1299 epoch rejection rate was below 10%,
supporting its use in EEG devices.
MRCPs are slow moving potentials that start before the movement onset and continue during
and after the movement [
36
]. In this research, MRCPs were recorded in motor tasks undertaken
while sitting and standing. The fact that they are challenging to record and process as they are
found in the delta band and are highly susceptible to movement artefacts and other sources of noise
makes them an excellent test case for evaluating EEG devices. Based on the findings of this research,
it can be asserted that the ADS1299 is an excellent choice for recording MRCPs. This assertion is
supported by the equivalence of signal-to-noise ratio, time and amplitude of the negative peak, and
the cosine similarity, the waveform differences of the MRCP with that of the laboratory-based system.
Additionally, the topographic maps showed that the cortical distribution of the EEG was similar across
the two systems in both tasks and concordant with previous research [
26
,
36
]. These findings have
implications for the translation of MRCP-based BCI paradigms to real-world applications [39,53,54].
Reflecting on the differences between the MRCP in the two motor tasks highlighted some
important findings which have implications for researchers interested in MRCP-based brain computer
interfaces targeted toward both single joint movements such as dorsiflexion and functional movements
such as walking [
55
] or sitting/standing [
56
]. The signal-to-noise ratio was significantly lower in step
on/off, due to higher noise in baseline EEG and a smaller negative peak in the MRCP. The decrease in
the peak negative value in step on/off may be explained by the torque generated in the two tasks [
26
],
although other task parameters may have played a role as well. This is supported by the fact that the
time of the negative peak was the same across both tasks, although it had larger variability in step
on/off. The differences across the tasks were found in both systems, suggesting that the ADS1299 is
sensitive enough to discern these variances.
The findings of this research should be considered in light of a number of limitations. First, EEG
and sEMG data were not recorded simultaneously with both systems. Rather, a single system was
used in a given recording session in a randomised order on different days. This protocol is in line with
the research comparing different EEG devices using within-participant designs [
22
–
24
]. Second, the
performance of an EEG recording system depends not only on the amplification chip but also on the
quality of the electrode system and the design of the printed circuit board (PCB). Thus, the performance
evaluation of the ADS199 performed in this research should be interpreted in relation with the used
electrode system (Compumedics Neuroscan Quick-Cap) and the circuit board (OpenBCI Cyton V3-32
board). Third, the sample rate used for ADS1299 was 250 Hz and PGA gain was 24 as these settings
result in minimum input referred noise [
1
]. On the other hand, in agreement with past research, the
sampling rate was set at 500 Hz for the NuAmps [
25
–
29
]. To address this, data from both systems were
down-sampled to 125 Hz before analysis. Fourth, the features studied in this research only represented
the lower range (<40 Hz) of the EEG spectrum. Thus, further research is required to investigate the
quality of EEG recordings with ADS1299 in the higher frequency bands.

Sensors 2018,18, 3721 13 of 18
5. Conclusions
This study has comprehensively demonstrated that the ADS1299 records low frequency (<40 Hz)
EEG at a level comparable to a laboratory-based system. Using a robust experimental design with
pre-planned statistical analysis, we found no significant differences across the two systems in both
EEG specific measures, such as power across bands, power ratio across bands, and pre-movement
noise, and MRCP specific measures, such as signal-to-noise ratio as well as time and amplitude of the
negative peak. In addition, this study illustrated differences in the MRCP of the two motor tasks, one
single joint, and the other multi-joint. We conclude that ADS1299 in combination with wet Ag/AgCl
electrodes is a good choice for both clinical research and the development of mobile BCIs based on low
frequency (<40 Hz) EEG.
Supplementary Materials:
The git difference report between the modified firmware used for the prototype and
version 3.0.0 of OpenBCI is available at http://www.mdpi.com/1424-8220/18/11/3721/s1. The data processing
and statistical analysis pipeline is available on GitHub at https://github.com/GallVp/evalMRCP.
Author Contributions:
U.R., I.K.N., N.S., and D.T. were involved in conceptualisation, the designing methodology
of the study, funding acquisition, and project administration. U.R. was involved in data curation, building software
pipeline, performing formal analysis, visualisation of results, and preparing the original draft. I.K.N., N.S., and D.T.
were involved in supervision. All authors were involved in reviewing and editing the manuscript.
Funding:
This research was funded by Medical Technologies Centre of Research Excellence (MedTech CoRE),
and Callaghan Innovation, New Zealand.
Acknowledgments:
We would like to acknowledge the advice given by Associate Professor Alain C. Vandal
(Auckland University of Technology, New Zealand) on statistical analysis methods.
Conflicts of Interest:
The authors declare no conflict of interest. The founding sponsors had no role in the design
of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the
decision to publish the results.
Abbreviations
The following abbreviations are used in this manuscript:
BCI Brain computer interface
dB Decibels
EEG Electroencephalography
GS Gold standard system based on Compumedics Neuroscan NuAmps
ICA Independent component analysis
MRCPs Movement-related cortical potentials
ms Milli-seconds
Proto The prototype system based on Texas Instruments ADS1299 and OpenBCI Cyton Board V3-32
PN Peak negative value in an MRCP
PNT Time of PN value
PMN Pre-movement noise measured as RMS value
RMS Root mean square value
r Cosine similarity
SNR Signal-to-noise ratio
sEMG Surface electromyography
S/D Participant wise ratio of power values in step on/off to those in dorsiflexion
µV, uV Micro-volts
µVrms, uVrms Micro-volts measured as RMS value

Sensors 2018,18, 3721 14 of 18
Appendix A
-4.1
0.7
r = 0.95
-1.5
0.1
r = 0.66
-4.9
2.5
r = 0.85
-2.4
1.3
r = 0.89
-3.3
0.8
r = 0.81
-7.1
3.9
r = 0.48
-6.2
1.8
r = 0.94
-3.9
2.1
r = 0.97
-5.5
0.7
r = 0.97
-1.9
1.7
r = 0.79
-1.8
0.9
r = 0.86
-2
0.2
r = 0.69
-8.5
3.7
r = 0.97
-4
2.1
r = 0.78
-9.8
3.2
r = 0.90
-2.6
1
r = 0.95
-4.6
1.1
r = 0.77
-5.3
1.5
r = 0.97
-3 -2 -1 0 1 2 3
Time (s)
-2.1
0.7
Amp. (uV)
r = 0.92
-3 -2 -1 0 1 2 3
Time (s)
-1.6
1.8
r = 0.54
-3 -2 -1 0 1 2 3
Time (s)
-4.5
1
r = 0.89
GS Proto
Figure A1.
Averaged MRCPs for 21 participants in dorsiflexion from gold standard and the prototype
along with cosine similarity (r). GS and Proto stand for the gold standard and the prototype
system respectively.

Sensors 2018,18, 3721 15 of 18
-2.7
1.9
r = 0.94
-0.8
0
r = -0.22
-5.2
3.8
r = 0.88
-1.4
2
r = 0.93
-1
1
r = 0.40
-3.7
3.6
r = 0.97
-4.3
3.4
r = 0.84
-3.6
3.5
r = 0.96
-2.7
1.5
r = 0.82
-0.9
1.2
r = 0.90
-1.4
0.9
r = 0.62
-0.7
0.9
r = 0.69
-4
3.2
r = 0.78
-1.1
0.8
r = 0.85
-7.4
5.1
r = 0.32
-2.7
1.6
r = 0.93
-5.6
3
r = 0.93
-2.3
1.5
r = 0.88
-3 -2 -1 0 1 2 3
Time (s)
-2.1
1.1
Amp. (uV)
r = 0.96
-3 -2 -1 0 1 2 3
Time (s)
-1.4
0.5
r = 0.65
-3 -2 -1 0 1 2 3
Time (s)
-3.4
3.6
r = 0.71
GS Proto
Figure A2.
Averaged MRCPs for 21 participants in step on/off from gold standard and the prototype
along with cosine similarity (r). GS and Proto stand for the gold standard and the prototype
system respectively.
References
1.
Instruments, T. Low-Noise, 8-Channel, 24-Bit Analog Front-End for Biopotential Measurements ADS1299; ADS1299
Datasheet; Texas Instruments Inc.: Dallas, TX, USA, 2012.
2. Krachunov, S.; Casson, A.J. 3D printed dry EEG electrodes. Sensors 2016,16, 1635. [CrossRef] [PubMed]
3.
Guerrero, F.N.; Spinelli, E.M. A Two-Wired Ultra-High Input Impedance Active Electrode. IEEE Trans.
Biomed. Circuits Syst. 2018,12, 437–445. [CrossRef] [PubMed]
4.
Nathan, V.; Jafari, R. Design principles and dynamic front end reconfiguration for low noise EEG acquisition
with finger based dry electrodes. IEEE Trans. Biomed. Circuits Syst. 2015,9, 631–640. [CrossRef] [PubMed]
5.
Zou, Y.; Nathan, V.; Jafari, R. Automatic identification of artifact-related independent components for artifact
removal in EEG recordings. IEEE J. Biomed. Health Inf. 2016,20, 73–81. [CrossRef] [PubMed]
6.
Qian, X.; Chen, Y.; Feng, Y.; Ma, B.; Hao, H.; Li, L. A method for removal of deep brain stimulation artifact
from local field potentials. IEEE Trans. Neural Syst. Rehabil. Eng. 2017,25, 2217–2226. [CrossRef] [PubMed]

Sensors 2018,18, 3721 16 of 18
7.
Kaongoen, N.; Yu, M.; Jo, S. Two-Factor Authentication System Using P300 Response to a Sequence of
Human Photographs. IEEE Trans. Syst. Man Cybern. Syst. 2017. [CrossRef]
8.
Beauchene, C.; Abaid, N.; Moran, R.; Diana, R.A.; Leonessa, A. The effect of binaural beats on visuospatial
working memory and cortical connectivity. PLoS ONE 2016,11, e0166630. [CrossRef] [PubMed]
9.
Jindal, U.; Sood, M.; Dutta, A.; Chowdhury, S.R. Development of point of care testing device for
neurovascular coupling from simultaneous recording of EEG and NIRS during anodal transcranial direct
current stimulation. IEEE J. Transl. Eng. Health Med. 2015,3, 1–12. [CrossRef] [PubMed]
10.
Zhao, X.; Zhao, D.; Wang, X.; Hou, X. A SSVEP Stimuli Encoding Method Using Trinary Frequency-Shift
Keying Encoded SSVEP (TFSK-SSVEP). Front. Hum. Neurosci. 2017,11, 278. [CrossRef] [PubMed]
11.
Ssentongo, P.; Robuccio, A.E.; Thuku, G.; Sim, D.G.; Nabi, A.; Bahari, F.; Shanmugasundaram, B.;
Billard, M.W.
; Geronimo, A.; Short, K.W. A Murine Model to Study Epilepsy and SUDEP Induced by
Malaria Infection. Sci. Rep. 2017,7, 43652. [CrossRef] [PubMed]
12.
Kranczioch, C.; Zich, C.; Schierholz, I.; Sterr, A. Mobile EEG and its potential to promote the theory and
application of imagery-based motor rehabilitation. Int. J. Psychophysiol.
2014
,91, 10–15. [CrossRef] [PubMed]
13.
Wolpaw, J.R.; Birbaumer, N.; McFarland, D.J.; Pfurtscheller, G.; Vaughan, T.M. Brain–computer interfaces for
communication and control. Clin. Neurophysiol. 2002,113, 767–791. [CrossRef]
14.
Lalor, E.C.; Kelly, S.P.; Finucane, C.; Burke, R.; Smith, R.; Reilly, R.B.; Mcdarby, G. Steady-state VEP-based
brain-computer interface control in an immersive 3D gaming environment. EURASIP J. Adv. Signal Process.
2005,2005, 706906. [CrossRef]
15.
Uktveris, T.; Jusas, V. Development of a Modular Board for EEG Signal Acquisition. Sensors
2018
,18, 2140.
[CrossRef] [PubMed]
16.
Dang, X.; Kang, B.; Liu, X.; Cui, G. An Interactive Care System Based on a Depth Image and EEG for Aged
Patients with Dementia. J. Healthc. Eng. 2017,2017. [CrossRef] [PubMed]
17.
Von Lühmann, A.; Wabnitz, H.; Sander, T.; Müller, K.R. M3BA: A mobile, modular, multimodal biosignal
acquisition architecture for miniaturized EEG-NIRS-based hybrid BCI and monitoring. IEEE Trans.
Biomed. Eng. 2017,64, 1199–1210. [CrossRef] [PubMed]
18.
Mora, N.; De Munari, I.; Ciampolini, P.; Millán, J.D.R. Plug&Play Brain–Computer Interfaces for effective
Active and Assisted Living control. Med. Biol. Eng. Comput. 2017,55, 1339–1352. [CrossRef] [PubMed]
19.
Kaongoen, N.; Jo, S. A novel hybrid auditory BCI paradigm combining ASSR and P300. J. Neurosci. Methods
2017,279, 44–51. [CrossRef] [PubMed]
20.
Belwafi, K.; Romain, O.; Gannouni, S.; Ghaffari, F.; Djemal, R.; Ouni, B. An embedded implementation based
on adaptive filter bank for brain-computer interface systems. J. Neurosci. Methods
2018
. [CrossRef] [PubMed]
21.
Jukiewicz, M.; Cysewska-Sobusiak, A. Stimuli design for SSVEP-based brain computer-interface.
Int. J. Electron. Telecommun. 2016,62, 109–113. [CrossRef]
22.
Oliveira, A.S.; Schlink, B.R.; Hairston, W.D.; König, P.; Ferris, D.P. Proposing metrics for benchmarking
novel EEG technologies towards real-world measurements. Front. Hum. Neurosci.
2016
,10, 188. [CrossRef]
[PubMed]
23.
Radüntz, T. Signal Quality Evaluation of Emerging EEG Devices. Front. Physiol.
2018
,9, 98. [CrossRef]
[PubMed]
24.
Ries, A.J.; Touryan, J.; Vettel, J.; McDowell, K.; Hairston, W.D. A comparison of electroencephalography
signals acquired from conventional and mobile systems. J. Neurosci. Neuroeng. 2014,3, 10–20. [CrossRef]
25.
Cravo, A.M.; Rohenkohl, G.; Wyart, V.; Nobre, A.C. Endogenous modulation of low frequency oscillations
by temporal expectations. J. Neurophysiol. 2011,106, 2964–2972. [CrossRef] [PubMed]
26.
Do Nascimento, O.F.; Nielsen, K.D.; Voigt, M. Relationship between plantar-flexor torque generation and
the magnitude of the movement-related potentials. Exp. Brain Res.
2005
,160, 154–165. [CrossRef] [PubMed]
27.
Gu, Y.; Dremstrup, K.; Farina, D. Single-trial discrimination of type and speed of wrist movements from
EEG recordings. Clin. Neurophysiol. 2009,120, 1596–1600. [CrossRef] [PubMed]
28.
Niazi, I.K.; Jiang, N.; Tiberghien, O.; Nielsen, J.F.; Dremstrup, K.; Farina, D. Detection of movement intention
from single-trial movement-related cortical potentials. J. Neural Eng. 2011,8, 066009. [CrossRef] [PubMed]
29.
Van Deursen, J.; Vuurman, E.; Smits, L.; Verhey, F.; Riedel, W. Response speed, contingent negative variation
and P300 in Alzheimer’s disease and MCI. Brain Cogn. 2009,69, 592–599. [CrossRef] [PubMed]
30.
Delorme, A.; Makeig, S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics
including independent component analysis. J. Neurosci. Methods 2004,134, 9–21. [CrossRef] [PubMed]

Sensors 2018,18, 3721 17 of 18
31.
Bonato, P.; D’Alessio, T.; Knaflitz, M. A statistical method for the measurement of muscle activation intervals
from surface myoelectric signal during gait. IEEE Trans. Biomed. Eng.
1998
,45, 287–299. [CrossRef] [PubMed]
32.
Ives, J.C.; Wigglesworth, J.K. Sampling rate effects on surface EMG timing and amplitude measures.
Clin. Biomech. 2003,18, 543–552. [CrossRef]
33.
Van Boxtel, G.; Geraats, L.; Van Den Berg-Lenssen, M.; Brunia, C. Detection of EMG onset in ERP research.
Psychophysiology 1993,30, 405–412. [CrossRef] [PubMed]
34.
Makeig, S.; Bell, A.J.; Jung, T.P.; Sejnowski, T.J. Independent component analysis of electroencephalographic
data. In Proceedings of the 8th International Conference on Neural Information Processing Systems,
27 November–2 December 1995; Cambridge, MA, USA; pp. 145–151.
35.
McFarland, D.J.; McCane, L.M.; David, S.V.; Wolpaw, J.R. Spatial filter selection for EEG-based
communication. Electroencephalogr. Clin. Neurophysiol. 1997,103, 386–394. [CrossRef]
36.
Shibasaki, H.; Hallett, M. What is the Bereitschaftspotential? Clin. Neurophysiol.
2006
,117, 2341–2356.
[CrossRef] [PubMed]
37.
De Vos, M.; Gandras, K.; Debener, S. Towards a truly mobile auditory brain–computer interface: Exploring
the P300 to take away. Int. J. Psychophysiol. 2014,91, 46–53. [CrossRef] [PubMed]
38.
Luck, S.J. An Introduction to the Event-Related Potential Technique; MIT Press: Cambridge, MA, USA, 2014;
Volume 1, pp. 110–115. [CrossRef]
39.
Jochumsen, M.; Christensen, C.R.; Christensen, H.R.; Cremoux, S.; Signal, N.; Allen, K.; Taylor, D.; Niazi, I.K.
Quantification of movement-related EEG correlates associated with motor training. Front. Hum. Neurosci.
2017,11. [CrossRef] [PubMed]
40.
Mrachacz-Kersting, N.; Kristensen, S.R.; Niazi, I.K.; Farina, D. Precise temporal association between
cortical potentials evoked by motor imagination and afference induces cortical plasticity. J. Physiol.
2012
,
590, 1669–1682. [CrossRef] [PubMed]
41.
Winter, B. Linear models and linear mixed effects models in R with linguistic applications. arXiv
2013
,
1, 7–18, arXiv:1308.5499.
42.
Bates, D.; Mächler, M.; Bolker, B.; Walker, S. Fitting Linear Mixed-Effects Models Using lme4. J. Stat. Softw.
2015,67, 1–48. [CrossRef]
43. Fox, J.; Weisberg, S. An R Companion to Applied Regression, 2nd ed.; Sage: Thousand Oaks, CA, USA, 2011.
44. Lenth, R.V. Least-Squares Means: The R Package lsmeans. J. Stat. Softw. 2016,69, 1–33. [CrossRef]
45.
Lakens, D. Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for
t-tests and ANOVAs. Front. Psychol. 2013,4, 863. [CrossRef] [PubMed]
46.
Dehak, N.; Dehak, R.; Glass, J.R.; Reynolds, D.A.; Kenny, P. Cosine similarity scoring without score
normalization techniques. In Proceedings of the Odyssey 2010: The Speaker and Language Recognition
Workshop, Brno, Czech Republic, 28 June–1 July 2010.
47.
Kamavuako, E.N.; Jochumsen, M.; Niazi, I.K.; Dremstrup, K. Comparison of features for movement
prediction from single-trial movement-related cortical potentials in healthy subjects and stroke patients.
Comput. Intell. Neurosci. 2015,2015, 71. [CrossRef] [PubMed]
48.
Ramos-Murguialday, A.; Birbaumer, N. Brain oscillatory signatures of motor tasks. J. Neurophysiol.
2015
,
113, 3663–3682. [CrossRef] [PubMed]
49.
Trammell, J.P.; MacRae, P.G.; Davis, G.; Bergstedt, D.; Anderson, A.E. The Relationship of Cognitive
Performance and the Theta-Alpha Power Ratio Is Age-Dependent: An EEG Study of Short Term Memory
and Reasoning during Task and Resting-State in Healthy Young and Old Adults. Front. Aging Neurosci.
2017
,
9, 364. [CrossRef] [PubMed]
50.
Symeonidou, E.R.; Nordin, A.D.; Hairston, W.D.; Ferris, D.P. Effects of Cable Sway, Electrode Surface
Area, and Electrode Mass on Electroencephalography Signal Quality during Motion. Sensors
2018
,18, 1073.
[CrossRef] [PubMed]
51.
Mihajlovi´c, V.; Patki, S.; Grundlehner, B. The impact of head movements on EEG and contact impedance: An
adaptive filtering solution for motion artifact reduction. In Proceedings of the 2014 36th Annual International
Conference of the IEEE Engineering in Medicine and Biology Society, Chicago, IL, USA, 26–30 August 2014;
pp. 5064–5067. [CrossRef]
52.
O’Regan, S.; Faul, S.; Marnane, W. Automatic detection of EEG artefacts arising from head movements using
EEG and gyroscope signals. Med. Eng. Phys. 2013,35, 867–874. [CrossRef] [PubMed]

Sensors 2018,18, 3721 18 of 18
53.
Niazi, I.K.; Mrachacz-Kersting, N.; Jiang, N.; Dremstrup, K.; Farina, D. Peripheral electrical stimulation
triggered by self-paced detection of motor intention enhances motor evoked potentials. IEEE Trans. Neural
Syst. Rehabil. Eng. 2012,20, 595–604. [CrossRef] [PubMed]
54.
Xu, R.; Jiang, N.; Dosen, S.; Lin, C.; Mrachacz-Kersting, N.; Dremstrup, K.; Farina, D. Endogenous sensory
discrimination and selection by a fast brain switch for a high transfer rate brain-computer interface.
IEEE Trans. Neural Syst. Rehabil. Eng. 2016,24, 901–910. [CrossRef] [PubMed]
55.
Sburlea, A.I.; Montesano, L.; de la Cuerda, R.C.; Diego, I.M.A.; Miangolarra-Page, J.C.; Minguez, J. Detecting
intention to walk in stroke patients from pre-movement EEG correlates. J. Neuroeng. Rehabil.
2015
,12, 113.
[CrossRef] [PubMed]
56.
Bulea, T.C.; Prasad, S.; Kilicarslan, A.; Contreras-Vidal, J.L. Sitting and standing intention can be decoded
from scalp EEG recorded prior to movement execution. Front. Neurosci. 2014,8, 376. [CrossRef] [PubMed]
©
2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).