ArticlePDF Available

Abstract and Figures

To facilitate the performance comparison of new methods for sleep patterns analysis, datasets with quality content, publicly-available, are very important and useful. We introduce an open-access comprehensive sleep dataset, called ISRUC-Sleep. The data were obtained from human adults, including healthy subjects, subjects with sleep disorders, and subjects under the effect of sleep medication. Each recording was randomly selected between PSG recordings that were acquired by the Sleep Medicine Centre of the Hospital of Coimbra University (CHUC). The dataset comprises three groups of data: (1) data concerning 100 subjects, with one recording session per subject; (2) data gathered from 8 subjects; two recording sessions were performed per subject, and (3) data collected from one recording session related to 10 healthy subjects. The polysomnography (PSG) recordings, associated with each subject, were visually scored by two human experts. Comparing the existing sleep-related public datasets, ISRUC-Sleep provides data of a reasonable number of subjects with different characteristics such as: data useful for studies involving changes in the PSG signals over time; and data of healthy subjects useful for studies involving comparison of healthy subjects with the patients, suffering from sleep disorders. This dataset was created aiming to complement existing datasets by providing easy-to-apply data collection with some characteristics not covered yet. ISRUC-Sleep can be useful for analysis of new contributions: (i) in biomedical signal processing; (ii) in development of ASSC methods; and (iii) on sleep physiology studies. To evaluate and compare new contributions, which use this dataset as a benchmark, results of applying a subject-independent automatic sleep stage classification (ASSC) method on ISRUC-Sleep dataset are presented.
Content may be subject to copyright.
c
o
m
p
u
t
e
r
m
e
t
h
o
d
s
a
n
d
p
r
o
g
r
a
m
s
i
n
b
i
o
m
e
d
i
c
i
n
e
1
2
4
(
2
0
1
6
)
180–192
j
o
ur
na
l
ho
me
pag
e:
www.intl.elsevierhealth.com/journals/cmpb
ISRUC-Sleep:
A
comprehensive
public
dataset
for
sleep
researchers
Sirvan
Khalighia,,
Teresa
Sousaa,
José
Moutinho
Santosb,
Urbano
Nunesa
aInstitute
of
Systems
and
Robotics
(ISR-UC),
Department
of
Electrical
and
Computer
Engineering,
University
of
Coimbra,
Portugal
bSleep
Medicine
Centre,
The
Central
Hospital
of
University
of
Coimbra
(CHUC),
Portugal
a
r
t
i
c
l
e
i
n
f
o
Article
history:
Received
6
May
2015
Received
in
revised
form
6
September
2015
Accepted
5
October
2015
Keywords:
Sleep
dataset
Automatic
sleep
stage
classification
Polysomnographic
signals
Effects
of
sleep
disorder
Medication
effects
Feature
selection
a
b
s
t
r
a
c
t
To
facilitate
the
performance
comparison
of
new
methods
for
sleep
patterns
analysis,
datasets
with
quality
content,
publicly-available,
are
very
important
and
useful.
We
introduce
an
open-access
comprehensive
sleep
dataset,
called
ISRUC-Sleep.
The
data
were
obtained
from
human
adults,
including
healthy
subjects,
subjects
with
sleep
disorders,
and
subjects
under
the
effect
of
sleep
medication.
Each
recording
was
randomly
selected
between
PSG
recordings
that
were
acquired
by
the
Sleep
Medicine
Centre
of
the
Hospital
of
Coimbra
University
(CHUC).
The
dataset
comprises
three
groups
of
data:
(1)
data
concerning
100
subjects,
with
one
recording
session
per
subject;
(2)
data
gathered
from
8
subjects;
two
recording
sessions
were
performed
per
subject,
and
(3)
data
collected
from
one
recording
session
related
to
10
healthy
subjects.
The
polysomnography
(PSG)
recordings,
associated
with
each
subject,
were
visually
scored
by
two
human
experts.
Comparing
the
existing
sleep-related
public
datasets,
ISRUC-Sleep
provides
data
of
a
rea-
sonable
number
of
subjects
with
different
characteristics
such
as:
data
useful
for
studies
involving
changes
in
the
PSG
signals
over
time;
and
data
of
healthy
subjects
useful
for
studies
involving
comparison
of
healthy
subjects
with
the
patients,
suffering
from
sleep
disorders.
This
dataset
was
created
aiming
to
complement
existing
datasets
by
providing
easy-to-
apply
data
collection
with
some
characteristics
not
covered
yet.
ISRUC-Sleep
can
be
useful
for
analysis
of
new
contributions:
(i)
in
biomedical
signal
processing;
(ii)
in
development
of
ASSC
methods;
and
(iii)
on
sleep
physiology
studies.
To
evaluate
and
compare
new
contrib-
utions,
which
use
this
dataset
as
a
benchmark,
results
of
applying
a
subject-independent
automatic
sleep
stage
classification
(ASSC)
method
on
ISRUC-Sleep
dataset
are
presented.
©
2015
Elsevier
Ireland
Ltd.
All
rights
reserved.
1.
Introduction
Sleep
is
an
active
and
regulated
process
with
an
essential
restorative
function
for
physical
and
mental
health
[1].
Quality
Corresponding
author.
Tel.:
+351
239796201.
E-mail
address:
skhalighi@isr.uc.pt
(S.
Khalighi).
of
sleep
and
sleep
disorders
have
an
important
effect
on
the
health
and
quality
of
life.
The
study
of
individual
behaviors
during
sleep
through
all-night
PSG
recordings
has
consistently
been
an
important
research
topic.
http://dx.doi.org/10.1016/j.cmpb.2015.10.013
0169-2607/©
2015
Elsevier
Ireland
Ltd.
All
rights
reserved.
c
o
m
p
u
t
e
r
m
e
t
h
o
d
s
a
n
d
p
r
o
g
r
a
m
s
i
n
b
i
o
m
e
d
i
c
i
n
e
1
2
4
(
2
0
1
6
)
180–192
181
Numerous
methods
have
been
developed
for
automatic
detection
of
arousals,
apnea,
and
sleep
stages
[2–5].
These
methods
often
use
PSG
recordings,
including
electrophys-
iological
signals
(electrocardiographic
activity,
brain-wave
patterns,
eye
movements,
and
activation
signal
of
mus-
cles),
pneumological
signals
(airflow,
blood
oxygen
level,
and
movement
of
respiratory
muscles),
and
other
contextual
infor-
mation
(body
position,
lights,
snore
recording,
etc.)
[6].
These
signals
have
been
collected
from
human
individuals
using
noninvasive
surface
electrodes.
To
evaluate
the
efficiency
of
automatic
sleep
pattern
analysis
methods,
non-public
and
few
existing
public
datasets
have
been
used.
Rigorous
compar-
isons
between
the
developed
methods
cannot
be
done
since
the
used
datasets
differ
in
recording
conditions,
physiolog-
ical
conditions
of
subjects
and
number
of
assessed
subjects.
Datasets
with
quality
content,
publicly
available,
are
an
impor-
tant
vehicle
for
accelerating
research,
since
they
facilitate
the
performance
comparison
of
new
approaches
and
methods.
This
paper
presents
three
main
contributions:
We
introduce
a
publicly-available
comprehensive
sleep
dataset,
called
ISRUC-Sleep,
which
comprises
three
sub-
groups
as
illustrated
in
Fig.
1.
The
subgroups
of
the
dataset
contain
PSG
signals
of
different
adult
individuals,
including
healthy
subjects,
subjects
with
sleep
disorders,
and
sub-
jects
under
the
effect
of
sleep
medication.
Sleep
stages
were
labeled
by
two
sleep
experts.
Furthermore,
for
8
sub-
jects
(subgroup-II),
two
sets
of
PSG
data,
which
have
been
recorded
at
different
time
dates,
are
provided.
Aiming
to
help
sleep
researchers
in
their
analysis
and
inferences
using
this
dataset,
for
each
subject,
useful
and
complementary
information
related
to
sleep
disorders,
used
medications,
and
their
effect
on
sleep
patterns,
are
pre-
sented.
Aiming
to
evaluate
and
compare
of
new
contributions,
which
will
use
this
dataset
as
a
benchmark,
results
of
applying
a
subject-independent
ASSC
method
on
ISRUC-
Sleep
dataset
are
presented.
This
supervised-learning
based
method,
detailed
in
Khalighi
et
al.
[7],
is
henceforth
named
SSM4S.1
2.
Terminology
and
definitions
Background
material,
terms
definition
(Table
1),
and
effects
of
sleep
disorders
and
medications
on
sleep
patterns
are
sum-
marized
in
the
next
subsections.
2.1.
Background
The
Rechtschaffen
and
Kales
standard
(R&K)
rules
are
the
basis
of
a
consensus
scoring
procedure
for
adults
[8].
The
American
academy
of
sleep
medicine
(AASM)
defined
new
criteria
for
sleep
scoring
based
on
the
R&K
rules.
In
adults,
sleep-wake
cycle
is
categorized
in
awake,
non-rapid
eye
movement
(NREM)
and
rapid
eye
movement
(REM)
sleep
stages.
NREM
sleep
is
further
divided
into
three
stages:
1Sirvan
Supervised
Method
for
Sleep
Staging.
N1
(drowsiness/transitional
sleep),
N2
(light
sleep)
and
N3
(deep
sleep)
[9],
the
last
of
which
is
also
called
delta
sleep
or
slow
wave
sleep
(SWS).
The
2007
AASM
visual
scoring
rules
recommend
a
frontal
electrode
for
best
detecting
K-
complexes,
a
central
electrode
for
spindles,
and
an
occipital
electrode
for
alpha
waves
[10].
Based
on
both
scoring
rule-
sets
(R&K
and
AASM),
epochs
of
30
s
(more
rarely
20
s)
are
defined
for
the
PSG
signals
scoring
[11].
Difficulties
in
sleep
scoring
arise
when
sleep
does
not
behave
in
accordance
with
the
normal/expected
way
as
a
consequence
of
sleep
disorders,
medication
or
in
face
of
individual
specific
char-
acteristics
of
sleep
Electroencephalogram
(EEG).
Since
the
collected
PSG
signals
are
characterized
by
low
signal-to-noise
ratio
(SNR),
a
preprocessing
stage
is
applied
to
improve
the
quality
of
the
signals;
i.e.
some
channels
of
the
recorded
signals
are
filtered
to
eliminate
noise
and
undesired
background
EEG,
aiming
to
enhance
the
PSG
signal
quality
and
increase
the
SNR.
The
filtering
stage
com-
prises:
(1)
a
notch
filter
to
eliminate
the
50
Hz
electrical
noise;
(2)
a
bandpass
Butterworth
filter
with
a
lower
cutoff
of
frequency
0.3
Hz
and
higher
cutoff
of
frequency
35
Hz
for
EEG
and
EOG
channels,
and
a
lower
cutoff
of
frequency
10
Hz
and
higher
cutoff
of
frequency
70
Hz
for
EMG
channels.
More
details
are
presented
in
Table
3.
The
common
EEG
frequency
bands
are:
low
delta
0.3–1
Hz,
delta
1–4
Hz,
theta
4–8
Hz,
alpha
8–12
Hz,
sigma
12–15
Hz,
and
beta
15–30
Hz.
Different
EEG
waves
(alpha,
beta,
sigma,
delta,
and
theta)
characterize
different
sleep
stages.
Low
amplitude,
mixed
EEG
frequency,
saw-tooth
pattern,
low
amplitude
Electromyogram
(EMG)
and
high
level
Electroocu-
logram
(EOG)
signals
from
both
eyes,
are
apparent
during
the
REM
stage.
In
stage
N1,
waves
with
high
amplitude
and
frequency
range
of
2–7
Hz
together
with
the
existence
of
alpha
waves
are
found
in
EEG
signal.
Still
regarding
N1,
EMG
level
is
lower
when
compared
to
the
awake
stage.
Sleep
spindles
(12–14
Hz)
and
K-complexes
are
observed
during
N2.
N3
(deep
sleep)
consists
of
low-frequency
high-
amplitude
waves
with
frequencies
of
2–4
Hz.
2.2.
Effect
of
sleep
related
disorders
and
sleep
pathology
on
sleep
stage
patterns
Sleep
apnea
is
the
most
frequent
sleep
disorder
seen
in
sleep
medicine
centers.
The
syndrome
is
characterized
by
repetitive
episodes
of
upper
airway
obstruction
that
occur
during
sleep
and
are
usually
associated
with
a
reduction
in
blood
oxygen
saturation.
These
nocturnal
respiratory
disturbances
result
in
brief
arousals
in
sleep,
which
promotes
sleep
fragmenta-
tion
that
typically
disturbs
sleep
architecture
with
reduction
or
even
complete
deprivation
of
REM
sleep
and
N3
sleep.
An
increase
of
arousals
of
different
length
together
with
an
increase
in
sleep
stage
changes
is
a
feature
of
the
syndrome.
This
fragmentation
of
sleep,
inhibiting
cortical
synchroniza-
tion,
would
be
responsible
for
the
lower
amount
of
slow
wave
sequences
of
the
deep
sleep
[12].
On
the
other
hand,
transient
experimental
hypoxia
induced
abnormal
posterior
resting
state
delta
and
alpha
rhythms
in
healthy
volunteers,
and
EEG
slowing
during
awake
with
an
increase
in
relative
theta
and
delta
power
in
occipital,
temporal
and
parietal
areas
182
c
o
m
p
u
t
e
r
m
e
t
h
o
d
s
a
n
d
p
r
o
g
r
a
m
s
i
n
b
i
o
m
e
d
i
c
i
n
e
1
2
4
(
2
0
1
6
)
180–192
Fig.
1
Details
of
ISRUC-Sleep
dataset.
was
observed
in
sleep
apnea
subjects
[13],
which
can
be
cor-
related
with
sleepiness
in
these
patients.
Also,
moderate
obstructive
sleep
apnea
(OSA)
patients
have
a
lower
percentage
of
slow
spindles
with
decelera-
tion
compared
to
mild
OSA
or
normal
groups
in
frontal
and
parietal
regions,
which
may
represent
a
disruption
of
thalamo-cortical
loops
generating
spindle
oscillations
[14].
All
together,
these
findings
can
contribute
to
significant
differences
in
agreement
among
multiple
raters
of
sleep
studies
in
sleep
apnea
patients
[15].
Affective
disorders
(depression/anxiety
disorder)
can
induce
sleep-EEG
changes.
In
depression,
an
increase
in
sleep
latency
and
an
increase
in
sleep
fragmentation
by
arousals
or
intermittent
awakenings,
and
early-morning
awakening
can
be
seen.
Frequently
a
shortened
REM
latency
including
sleep
onset
REM
periods
(REM
latency
<20
min),
prolonged
first
REM
periods
with
an
increase
of
REM
density
is
present
in
all
age
groups
[16];
NREM-sleep
changes
include
a
decreased
SWS,
EEG
delta
power
throughout
the
night
and
increase
of
stage
N2
sleep
together
with
a
shift
of
EEG-delta
power
from
the
first
to
the
second
sleep
cycle
in
younger
patients
[17].
In
gen-
eralized
anxiety
disorder
(GAD)
patients
no
clear
differences
with
control
subjects
in
SWS
and
REM
sleep
are
seen,
being
the
differences
confined
to
insomnia-like
symptoms
[18].
Never-
theless,
some
studies
have
shown
increased
REM
latency
[19],
or
increased
mean
REM
latency
over
consecutive
nights
[20],
in
GAD
patients
compared
to
depressed
patients,
which
could
be
useful
to
distinguish
both
disorders.
In
primary
insomnia
(psychophysiological
insomnia)
the
hyperarousal
model
suggests
that
a
deficit
of
reduce
arousal
during
sleep
may
be
responsible
for
non-restorative
sleep.
It
has
been
shown
that
in
these
patients
there
is
elevated
spectral
power
values
in
the
EEG
beta
(cortical
arousal)
and
sigma
(spindle)
frequency
band
during
N2
sleep
stage
with
no
differences
in
other
frequency
bands.
This
increase
in
cortical
arousal
and
in
an
index
of
sleep
protective
mecha-
nisms
(spindles)
may
provide
further
evidence
for
the
concept
that
a
simultaneous
activation
of
wake-promoting
and
sleep-protecting
neural
activity
patterns
contributes
to
the
experience
of
non-restorative
sleep
in
primary
insomnia
[21].
In
dementia,
sleep
is
characteristically
grossly
disrupted
with
lower
sleep
efficiency,
higher
percentage
of
N1
and
increase
of
arousals
and
awakenings.
A
decrease
of
SWS
can
be
expected
but
the
hallmark
of
the
sleep
EEG
in
these
patients
is
a
slowing
EEG
activity
with
spindles
reduction.
Therefore,
the
scoring
of
sleep
stages
may
be
challenging
[18].
2.3.
Effect
of
medications
on
sleep
stage
patterns
The
medication
can
also
affect
the
EEG
sleep
patterns.
Chronic
use
of
benzodiazepines
has
shown
to
induce
an
increase
of
Table
1
List
of
acronyms.
Acronym
Description
Acronym
Description
AASM
The
American
Academy
of
Sleep
Medicine
LOOCV
Leave-one
subject-out
cross-validation
ACC
Accuracy
MODWT
Maximal
overlap
discrete
wavelet
transform
AR
Auto
regressive
NRI
Noradrenaline
reuptake
inhibitor
ASSC
Automatic
sleep
stage
classification
NREM
Non-rapid
eye
movement
BCR
Balanced
classification
rate
PSG
Polysomnography
CAP
The
cyclic
alternating
pattern
REM
Rapid
eye
movement
CV
Cross
validation
R&K
Rechtschaffen
and
Kales
standard
ECG
Electrocardiogram
SAS
Sleep
apnea
syndrome
EEG
Electroencephalogram
SENS
Sensitivity
EMG
Electromyogram
SNR
Signal-to-noise
ratio
EOG
Electrooculogram
SNRI
Selective
noradrenaline
reuptake
inhibitors
GAD
Generalized
anxiety
disorders
SPEC
Specificity
ISRUC
Institute
of
Systems
and
Robotics,
University
of
Coimbra
SSRI
Selective
serotonin
reuptake
inhibitors
SWS
Slow
wave
sleep
c
o
m
p
u
t
e
r
m
e
t
h
o
d
s
a
n
d
p
r
o
g
r
a
m
s
i
n
b
i
o
m
e
d
i
c
i
n
e
1
2
4
(
2
0
1
6
)
180–192
183
sleep
stage
N2,
decrease
of
N3
(lower
delta
activity
and
theta
activity),
and
an
increase
of
arousals
[22];
increase
of
spindle
activity
that
can
intrude
in
REM
sleep
is
also
described
[23].
REM
sleep
reduction
or
even
complete
suppression
was
reported
in
humans
after
an
administration
of
tri-
cyclic
or
tetracyclic
antidepressants
[24],
monoaminoxidase
inhibitors
[25,26],
SSRIs
[27],
selective
noradrenaline
reuptake
inhibitors
(NRI)
[28],
SSRIs
and
selective
noradrenaline
reup-
take
inhibitors
(SNRI)
[29].
A
few
exception
should
be
the
noradrenergic
and
specific
serotoninergic
antidepressant
mir-
tazapine
[30]
which
also
increase
total
sleep
time
and
sleep
efficiency
after
four
weeks
of
administration.
Most
tricyclic
antidepressants
increase
SWS
[24],
but
there
is
evidence
that
selective
serotonin
reuptake
inhibitors
(SSRI)
impairs
sleep
continuity
by
increasing
of
intermittent
wake-
fulness
[18].
On
the
other
side
an
increase
of
sleep
stage
N3
was
found
during
treatment
of
depressed
patients
with
the
SNRI
duloxetine
[29]
and
a
decrease
in
REM
sleep
and
increase
in
REM
sleep
latency
were
observed
with
the
SNRI
venlafaxine
[31].
An
increase
in
the
total
sleep
time
and
sleep
efficiency,
and
a
decrease
in
the
time
spent
awake,
were
verified
in
patients
with
depression,
under
mirtazapine
med-
ication.
These
changes
persisted
after
four
weeks
[30].
Trazodone,
a
triazolopyridine
antidepressant
weak,
is
a
specific
inhibitor
of
serotonin
(5-HT)
reuptake.
The
use
of
this
medication
showed
increases
in
sleep
efficiency,
total
sleep
time,
total
sleep
period,
N3
and
REM
duration,
as
well
as
decreases
in
wakefulness
during
the
total
sleep
period,
early
morning
awakening,
and
N2
[32].
Besides
alterations
in
sleep
EEG
induced
by
diseases
or
medications
in
healthy
people,
there
is
specific
individual
sleep
patterns
that
can
be
seen
as
a
fingerprint
that
allows
a
correct
discrimination
between
individuals
with
a
probability
of
92%
[33].
This
EEG
fingerprint
is
genetically
determined
as
shown
by
studies
on
monozy-
gotic
and
dizygotic
twins,
particularly
in
the
range
of
alpha
and
sigma
bands
[33].
The
above
observations
can
explain
why
human-experts
analysis
of
sleep
EEG
can
be,
so
far,
superior
to
computer
anal-
ysis
by
a
better
adaptation
to
the
individual
characteristics
of
the
electrophysiologic
signals,
recognizing
specific
sleep-
related
characteristics
constructing
their
own,
subjective
and
patient
specific
PSG
pattern,
which
allows
to
decide
what
the
most
likely
sleep
stage
is
[23].
On
the
other
hand,
this
also
explains
why
as
much
as
25%
of
overall
disagreement
can
be
seen
between
two
human-experts
sleep
scorings
of
the
same
recording
[34].
The
disagreement
is
particularly
seen
in
N1
where
the
AASM
definition
include
attenuation
or
slowing
of
the
alpha
rhythm
and
the
presence
of
slow
eye
movements,
4–7
Hz
EEG
and
vertex
sharp
waves.
Many
subjects
do
not
gen-
erate
some
or
even
any
of
these
waves
as
is
stated
in
the
manual
[23].
3.
Sleep
datasets
To
assess
the
efficiency
of
sleep
pattern
analysis
methods,
each
research
team
collects
their
own
test
data
with
expendi-
ture
of
time
and/or
financial
resources
[47,48].
These
datasets,
mainly
used
in
the
context
of
their
own
research,
often
lack
several
relevant
information
details
regarding
acquisition
and
subject
pathological
conditions
(neural,
cardiorespiratory,
medication
effects).
Some
of
these
datasets
[49,50]
also
lack
statistical
significance
and
just
recorded
some
of
the
PSG
channels.
Therefore,
an
accurate
and
comparative
evaluation
of
the
performances
of
these
methods
with
new
methods
can-
not
be
done
effectively.
Recognizing
the
need
and
usefulness
of
publicly
available
sleep
datasets,
which
can
be
used
as
a
common
reference
for
researchers,
some
sleep-related
datasets
were
developed
by
sleep
research
groups.
As
shown
in
Table
2,
these
datasets
contain
multiple
signals
from
some
healthy
and
patient
sub-
jects.
The
sleep
datasets
of
PhysioBank
[35]
have
been
used
in
a
few
works
(see
Table
2).
Even
though
MIT-BIH,
Sleep-EDF
and
Extended
Sleep-EDF
are
general
purpose
datasets,
these
do
not
have
enough
subjects
for
generalization
purposes.
CAP-Sleep
dataset
is
an
exception
in
PhysioBank
repository,
containing
108
recordings,
however
it
consists
of
the
specific
data
useful
for
studies
related
to
CAP.
The
sleep
heart
health
study
(SHHS)
dataset
[43],
which
has
a
convenient
number
of
recordings,
is
not
a
completely
public
dataset.
It
is
available
only
upon
special
request
and
approval.
On
the
other
hand,
due
to
providing
just
the
signals
of
two
EEG
(C3-A2
and
C4-A1)
channels,
SHHS
has
limi-
tations
for
general-purpose
sleep
research.
In
fact,
it
is
a
specific
purpose
dataset
useful
in
research
studies
involving
relationships
between
sleep-disordered
breathing
and
heart
diseases.
Recently,
Montreal
archive
of
sleep
study
(MASS),
which
is
an
open-access
sleep
dataset
collected
from
healthy
subjects,
was
proposed
by
O’Reilly
et
al.
[45].
Although
it
is
reported
that
the
dataset
contains
data
of
200
participants,
it
is
a
col-
lection
of
five
different
subgroup
of
data.
These
subgroups
were
pooled
from
8
different
research
protocols
performed
in
3
different
hospital-based
sleep
laboratories.
Furthermore,
there
exist
some
access
restrictions
regarding
different
kinds
of
information
of
the
dataset.
As
detailed
in
Table
2,
the
sub-
groups
of
this
dataset
have
significant
differences
in
terms
of
number
of
channels,
filtering
methods
applied
to
the
signals,
acquisition
software,
annotations,
scoring
criteria
and
epoch
size.
In
summary,
all
the
dataset
detailed
in
Table
2
have
limita-
tions
in
some
aspects
and
as
far
as
we
know,
except
Sleep-EDF
dataset
(expanded),
which
were
recorded
in
two
subsequent
day-night
at
the
subject’s
home,
in
the
others
only
one
acqui-
sition
session
(one
recording)
per
subject
is
available.
4.
ISRUC-Sleep
dataset
ISRUC-Sleep
dataset
contains
data
collected
from
all-night
PSG
recordings
with
duration
around
eight
hours.
Each
recor-
ding
was
randomly
selected
between
PSG
recordings
that
were
acquired
by
the
Sleep
Medicine
Centre
of
the
Hospital
of
Coimbra
University
(CHUC),
in
the
period
2009–2013.
Over-
all
standard
setup
setting
for
data
acquisition,
comprised
a
biosignal
acquisition
equipment
(a
SomnoStar
Pro
sleep
sys-
tem
which
is
a
multi-channel
ambulatory
recording
device),
and
a
set
of
sensors
collecting
data
in
a
non-invasive
way,
according
to
the
international
10–20
standard
electrode
place-
ment
(Fig.
2)
[52].
With
regard
to
the
arrangements,
the
subject
184
c
o
m
p
u
t
e
r
m
e
t
h
o
d
s
a
n
d
p
r
o
g
r
a
m
s
i
n
b
i
o
m
e
d
i
c
i
n
e
1
2
4
(
2
0
1
6
)
180–192
Table
2
Some
of
the
public
sleep
datasets.
Dataset
Subjects/sampling
rate
Recorded
channels
Recording
duration
Purpose
of
creating
the
dataset
Subjects
age
Literatures
cited
dataset
MIT-BIH
[35] 18
recording
of
16
subjects
with
or
without
sleep
apnea
syn-
drome(SAS)/250
Hz
Four-,
six-,
and
seven-channel
recordings
of
ECG
signal,
an
invasive
blood
pressure
signal,
an
EEG
signal,
a
respiration
signal
and
a
text.
Some
records
contain
other
signals
such
as,
respiratory
effort
signal.,
an
EOG
signal,
an
EMG
(from
the
chin)
signal,
a
stroke
volume
signal
8–10
h General
purpose 32–56,
Avg.
=
43
Adnane
et
al.
[36],
Nicolaou
and
Georgiou
[37],
Fraiwan
et
al.
[38]
Sleep-EDF
[35]
8
subjects
without
any
sleep-related
medication,
scored
based
on
R&K/100
Hz
EEG
(Fpz-Cz
and
Pz-Oz),
Horizontal
EOG,
submental-EMG
envelope,
oro-nasal
airflow,
rectal
body
temperature
and
an
event
marker
1.25–6.5
h
General
purpose
21–35
Bajaj
and
Pachori
[4],
Ronzhina
et
al.
[39]
Expanded
Sleep-EDF
[35]
61
recordings
from
healthy
subjects,
without
any
sleep-related
medication/100
Hz
EEG(Fpz-Cz
and
Pz-Oz),
Horizontal
EOG,
submental
chin
EMG,
and
an
event
marker
Around
9
h
Study
of
age
effects
on
sleep;
study
of
temazepam
effect
on
sleep
25–101
Kemp
et
al.
[40],
Yaghouby
et
al.
[41]
CAP-Sleep
[35] 108
recordings
scored
based
on
R&K/512
Hz
3
EEG
channels
(F3
or
F4,
C3
or
C4
and
O1
or
O2,
referred
to
A1
or
A2),
2
EOG
channels,
EMG
of
the
submental
is
muscle,
bilateral
anterior
tibial
EMG,
respiration
signals
(airflow,
abdominal
and
thoracic
effort
and
SaO2)
and
ECG
8–10
h
Study
of
the
cyclic
alternating
pattern
(CAP)
30–75
Terzano
et
al.
[42]
SHHS-1,
-2
[43]
9736
recordings
scored
based
on
R&K/125
Hz
2
EEG
channels
(C3
or
C4,
referred
to
A1
or
A2),
2
EOG
channels,
EMG
of
the
submental,
bilateral
anterior
tibial
EMG,
respiration
signals
(airflow,
abdominal
and
thoracic
effort)
and
ECG
Overnight
Study
of
OSA,
sleep-disordered
breathing,
and
heart
diseases
40
and
older
Ebrahimi
et
al.
[44]
MASS
[45] Collection
of
200
recordings
(5
different
subsets)
scored
based
on
R&K
or
AASM/256
Hz
4–20
EEG
channels,
2–4
EOG
channels,
1–5
EMG
of
the
submental,
bilateral
anterior
tibial
EMG,
sometimes
with
respiration
signals
(airflow,
abdominal
and
thoracic
effort
and
SaO2)
and
ECG
Overnight Study
of
the
sleep
spindles
and
general
purpose
18–76
Tsanas
et
al.
[46]
c
o
m
p
u
t
e
r
m
e
t
h
o
d
s
a
n
d
p
r
o
g
r
a
m
s
i
n
b
i
o
m
e
d
i
c
i
n
e
1
2
4
(
2
0
1
6
)
180–192
185
Fig.
2
The
international
10–20
system
seen
from
(A)
left
and
(B)
above
the
head.
A
=
ear
lobe,
C
=
central,
Pg
=
nasopharyngeal,
P
=
parietal,
F
=
frontal,
Fp
=
frontal
polar,
O
=
occipital
[51].
sleeps
in
a
bed
in
a
patient’s
room,
and
the
experts
and
tech-
nicians
stay
in
a
separate
room.
All
patients
referred
were
submitted
to
an
initial
briefing
with
the
support
of
an
informed
consent
document.
The
ethics
committee
of
CHUC
approved
the
use
of
the
data
of
the
referred
patients
as
anonymous
for
the
research
purposes.
The
PSG
signals
were
recorded
according
to
the
recom-
mendations
of
the
AASM
manual.
As
described
in
Table
3,
each
recording
consists
of
signals
from
19
channels.
All
EEG,
EOG,
and
chin
EMG
signals
were
sampled
at
200
Hz
and
stored
using
the
standard
EDF+
data
formats
with
.REC
extension
[53].
All
recordings
of
the
dataset
were
segmented
into
epochs
of
30
s
and
visually
scored
by
two
different
sleep
experts
in
CHUC
according
to
the
guidelines
of
AASM
[9],
with
the
stages:
awake,
NREM
(N1,
N2,
and
N3)
and
REM
sleep.
Calculating
Cohen’s
kappa
index
between
two
experts
over
the
subjects
of
the
subgroups
yields
to
the
following
Kappa
indexes:
overall
kappa
index
of
0.87
±
0.09
for
subgroup-I,
0.82
±
0.15
for
subgroup-II,
and
0.9
±
0.06
for
subgroup-III.2The
labels
are
stored
in
standard
text
file
format,
where
each
line
corresponds
to
one
epoch;
More-
over,
the
gender,
height,
weight,
age
and
date
of
recording,
of
individuals
tested
are
recorded
in
the
header
of
each
text.
Further
analysis
such
as
sleep
events,
sleep
related
disor-
ders,
other
diseases,
sleep
pathology,
used
medications,
EEG
pattern
alterations,
and
percentage
of
each
sleep
stage
for
each
subject
are
presented.
ISRUC-Sleep
dataset
comprises
three
subgroups
of
data3as
described
in
Table
4.
2Distribution
of
the
individual
kappa
indexes
over
the
recordings
are
presented
in
the
result
files,
which
are
available
via
http://sleeptight.isr.uc.pt/ISRUC
Sleep/.
3Recordings,
summary
of
the
characteristics,
and
clinical
information
for
each
subject
of
the
dataset
are
available
via
http://sleeptight.isr.uc.pt/ISRUC
Sleep/.
5.
Application
of
ISRUC-Sleep
in
ASSC
Some
data
of
Subgroup-I
was
already
used
in
a
few
works
[7,54–57].
Aiming
to
improve
the
applicability
of
automatic
sleep
staging,
we
proposed
the
SSM4S
classification
method
with
main
blocks
depicted
in
Fig.
3.
In
this
ASSC
method,
after
applying
common
preprocessing,
and
segmentation
of
the
signals
in
30
s
epochs,
some
features
are
extracted
using
sev-
eral
methods
in
the
temporal,
frequency
and
time-frequency
domains.
PSG
signals
are
traditionally
analyzed
in
the
fre-
quency
domain,
since
each
sleep
stage
is
characterized
by
a
specific
pattern
of
frequency
contents.
Moreover,
PSG
signals
are
non-stationary;
therefore
time-frequency
transformations
like
wavelets
are
very
useful.
Due
to
superiority
of
the
maximal
overlap
discrete
wavelet
transform
(MODWT)
[7,58]
versus
dis-
crete
wavelet
transform,
a
MODWT
of
depth
6
with
Daubechies
order
four
(db4)
is
applied
to
every
30
s
epochs
with
a
sampling
rate
of
200
Hz.
The
frequency
ranges
are
broken
down
into
ı
range
(<4
Hz),
range
(4–8
Hz),
˛
range
(8–13
Hz)
and
ˇ
range
(13–30
Hz).
To
represent
the
time-frequency
distribution
of
the
EEG,
EOG
and
EMG
signals,
features
such
as
energy,
percent
of
energy
[55],
mean
and
standard
deviation
are
extracted
from
each
sub-band.
Furthermore,
due
to
the
importance
of
spectral
and
tem-
poral
analysis,
features
such
as
relative
spectral
power,
peak
to
peak
amplitude
of
two
EOGs,
Tsallis
(q
=
2),
Renyi
(˛
=
2),
Shannon
entropy,
Hjorth
parameters,
harmonic
parameters,
percentile
25,
50,
75,
autoregressive
coefficients
(order
3),
slow
wave
index
(SWI),
kurtosis
and
skewness
[1]
are
extracted
from
EEG
and
EOG
channels.
To
reduce
the
influence
of
extreme
values,
the
matrix
of
features
Y
is
transformed
as
follows:
X
=
arcsin(Y)
(1)
186
c
o
m
p
u
t
e
r
m
e
t
h
o
d
s
a
n
d
p
r
o
g
r
a
m
s
i
n
b
i
o
m
e
d
i
c
i
n
e
1
2
4
(
2
0
1
6
)
180–192
Table
3
Details
of
recorded
signals
of
ISRUC-Sleep
dataset.
Channel
number
Type
of
the
signal
Label
Frequency
rate/Hz
Butterworth
Notch
filter
Description
1EOG LOC-A2 200
0.3
Hz–35
Hz
50
Hz Left
eyes
movements
2
ROC-A1
Right
eyes
movements
3
EEG
F3-A2
200
0.3
Hz–35
Hz
50
Hz
Brain
channels
with
the
references
A1
and
A2,
which
placed
in
the
left
and
right
ear-lobes
4
C3-A2
5
O1-A2
6
F4-A1
7
C4-A1
8
O2-A1
9
Chin
EMG
X1
200
10
Hz–70
Hz
50
Hz
Chin
EMG,
placed
between
the
chin
and
the
lower
lip
10
ECG
(EKG)
X2
200
50
Hz
Electrocardiographic
11
Leg-1
EMG
X3 200
10
Hz–70
Hz
50
Hz Left
leg
movement
12
Leg-2
EMG
X4
Right
leg
movement
13
Snore
X5
200
10
Hz–70
Hz
50
Hz
Snore
(derived)
14
Flow-1
X6
12.5 Airflow
(pressure
based)
15
Flow-2
DC3
25
16 Abdominal X7 25 Abdominal
efforts
17
X8
18
Pulse
oximetry
SaO2
12.5
Pulse
oximetry
(SaO2)
19
Body
position DC8
25
Body
position
(BPOS)
where
Y
denotes
the
feature
matrix,
and
X
=
{xij;
i
=
1,
2,
.
.
.,
N
and
j
=
1,
2,
.
.
.,
M}
(2)
is
the
transformed
feature
matrix,
where
N
and
M
denote
the
number
of
subjects
and
the
number
of
features,
respectively,
and
then
to
avoid
features
in
greater
numeric
ranges
domi-
nating
those
in
smaller
numeric
ranges,
as
well
as
numerical
difficulties
during
classification;
each
feature
of
the
trans-
formed
matrix
X
is
independently
normalized
to
the
[0,1]
range
by
applying
xij =
xij/(max(xj)
min(xj))
(3)
where
xjis
a
vector
of
each
independent
feature.
Next,
a
two-
step
feature
selection
process
that
consists
on
a
filtering
and
a
wrapper
phases
is
performed:
firstly,
as
detailed
in
[7],
the
less
discriminative
feature-types
are
removed
and
features
such
as
relative
spectral
power,
harmonic
parameters,
percentile
75,
autoregressive
coefficients
(order
3),
kurtosis
and
skewness
[1]
are
selected.
Then,
in
the
second
step,
to
select
the
best
elements
of
each
feature-type,
resulted
feature
vector
is
fed
into
a
mRMR
feature
selector.
Finally
the
features
are
classified
based
on
classical
supervised
learning
by
a
SVM
classifier.
6.
Performance
evaluation
We
briefly
present
the
average
performances
of
SSM4S
method,
using
different
subgroups
of
the
ISRUC-Sleep
dataset.
Two
types
of
experiments
have
been
carried
out:
sleep-wake
detection
and
multiclass
sleep
staging
based
on
AASM.
The
purpose
of
these
experiments
is
to
provide
the
reader
with
sufficient
evidence
to
consider
ISRUC-Sleep
as
a
working
Table
4
Characteristics
of
ISRUC-Sleep
dataset.
Dataset
Subjects
Number
of
recording
per
subject
Subject
characteristics
Subjects
age
Subgroup-I
100
subjects
(55
male,
45
female)
with
evidence
of
having
sleep
disorders
One
data
acquisition
session
per
subject
Most
of
the
subjects
have
detected
sleep
apnea
events;
the
subjects
could
be
under
medication,
but
all
were
in
position
to
breathe
without
the
help
of
machine
20–85,
Avg.
=
51,
std.
=
16
years
Subgroup-II
8
subjects
(6
male,
2
female)
with
evidence
of
having
sleep
disorders
Two
data
acquisition
sessions
were
performed
in
two
different
dates
Detected
sleep
apnea
events;
the
subjects
could
be
under
medication,
but
all
were
in
position
to
breathe
without
the
help
of
machine
26–79,
Avg.
=
46.87,
std.
=
18.7
years
Subgroup-III
10
subjects
(9
male,
1
female)
One
data
acquisition
session
per
subject
Healthy
subjects
(control
group)
30–58,
Avg.
=
40,
std.
=
10
years
c
o
m
p
u
t
e
r
m
e
t
h
o
d
s
a
n
d
p
r
o
g
r
a
m
s
i
n
b
i
o
m
e
d
i
c
i
n
e
1
2
4
(
2
0
1
6
)
180–192
187
Fig.
3
Structure
of
the
automatic
sleep
stage
classification
method
SSM4S.
dataset
for
the
development
of
sleep
monitoring
and
analysis
systems.
To
evaluate
the
system
the
following
performance
measures
were
used:
accuracy
(ACC),
specificity
(SPEC),
sen-
sitivity
(SENS).
In
addition
to
these
common
performance
measures,
the
balanced
correction
rate
(BCR),
which
is
the
average
of
the
proportion
of
correct
classifications
in
each
class,
was
also
calculated.
The
ASSC
method
was
applied
on
data
of
subgroup-I,
-II
and
-III.
Due
to
the
high
number
of
subjects,
all
the
assess-
ments
with
the
data
of
subgroup-I,
were
determined
by
five-
and
ten-fold
cross
validation.
However,
since
there
are
eight
and
ten
subjects
in
subgroup-II
and
-III,
respectively,
to
ver-
ify
reliability
of
the
results,
all
the
experiments
with
these
two
subgroup
were
done
using
leave-one
subject-out
cross-
validation
(LOOCV)
strategy.
The
experimental
results
for
sleep-wake
detection
are
summarized
in
Table
5.
From
the
analysis
of
detailed
results,4it
was
verified
that,
the
higher-
performance
values
were
attained
with
the
subjects
with
longer
periods
of
awake
stage
during
the
all-night
recor-
ding
(approximately
8
h
of
data
collection).
As
expected,
the
ASSC
method
of
SSM4S,
achieved
worse
average
and
standard
deviation,
with
data
of
the
subjects
with
suspected
sleep
disor-
ders
(subgroup-I).
The
ambiguous
patterns
on
PSG
recordings,
mainly
due
to
sleep
disorders
and
artifacts,
can
affect
the
per-
formance
of
the
method.
Moreover,
the
recordings
of
subgroup-I,
-II
and
-III
were
used
to
evaluate
SSM4S,
in
multiclass
sleep
staging.
Based
on
average/std.,
shown
in
Table
6,
the
best
discrimination
were
achieved
for
awake,
N3,
REM,
and
N2
stages,
respectively.
The
4The
detailed
results
of
the
overall
performance
of
SSM4S,
associated
with
the
dataset
are
available
via
http://sleeptight.isr.uc.pt/ISRUC
Sleep/Results.
lowest
average
performance
resides
in
the
classification
of
stage
N1.
7.
Analysis
of
ISRUC-Sleep
dataset
for
ASSC
This
section
summarizes
the
main
conclusions
derived
from
applying
SSM4S
over
ISRUC-Sleep
dataset.
To
analyze
the
rela-
tion
of
experts
agreement
and
classification
performance
two
measures
were
calculated.
Auto-regressive
coefficients,
which
is
a
representation
of
a
time
series
such
that
it
specifies
that
output
variable
depends
linearly
on
its
own
previous
values,
and
balanced
correlation
rate
(BCR)
were
evaluated.
7.1.
For
sleep-wake
detection
There
is
a
remarkable
correlation
between
the
agreement
levels
of
two
experts
and
the
classification
performance
(Fig.
4).
Despite
of
this
inference,
a
few
exception
(recordings
related
to
subjects
12
and
40
of
100
subjects
of
subgroup-
I)
were
found
with
high
agreement
level
of
experts
and
Fig.
4
(a)
Distribution
of
balanced
classification
rate
(BCR)
values
concerning
two
experts
agreement
in
sleep-wake
visual
scoring;
(b)
distribution
of
BCR
values
corresponding
to
auto
regressive
(AR)-coefficient,
in
automatic
sleep-wake
detection.
188
c
o
m
p
u
t
e
r
m
e
t
h
o
d
s
a
n
d
p
r
o
g
r
a
m
s
i
n
b
i
o
m
e
d
i
c
i
n
e
1
2
4
(
2
0
1
6
)
180–192
Table
5
Average
results
of
the
ASSC
method
SSM4S
for
sleep-wake
detection.
Balanced
classification
rate
(BCR),
specificity
(SPEC),
sensitivity
(SENS),
accuracy
(ACC)
are
calculated
for
sleep
and
awake
stages.
Method
Applying
method
on
subgroup-I
Applying
method
on
subgroup-II
Applying
method
on
subgroup-III
Cross
validation Five-fold
CV Ten-fold
CV
LOOCV
LOOCV
Average
BCR
90.16
±
06.88
90.30
±
06.29
82.84
±
10.26
91.19
±
03.15
Average
SENS
83.98
±
14.99
84.13
±
14.67
69.11
±
23.07
85.03
±
07.05
Average
SPEC
96.34
±
06.88
96.47
±
03.50
96.57
±
05.12
97.35
±
01.37
Average
ACC
93.97
±
06.32
94.10
±
06.19
92.40
±
05.15
95.39
±
01.10
very
low
classification
performance.
Since
alpha
activity
is
one
of
the
relevant
patterns
in
awake
stage,
this
factor
can
affect
the
performance
of
awake
detection.
The
observed
low
amplitude
of
the
alpha
activity
in
the
EEG
signals
of
sub-
ject
12
of
subgroup-I
can
be
the
main
reason
of
performance
degradation.
For
subject
40
of
subgroup-I,
the
artifacts
in
EEG
signals,
which
resulted
from
the
low
quality
of
data
acquisition,
affected
the
classification
performance.
There
is
also
a
correlation
between
degradation
of
the
clas-
sification
performance,
and
the
increase
of
the
number
of
arousals
and
awakens.
There
is
none
significant
relation
between
characteristics
such
as
age,
gender,
diagnosis
and
medication,
and
the
clas-
sification
performance.
Unusual
patterns
of
alpha
activity
and
rapid
activity
affected
the
classification
performance.
7.2.
For
multiclass
sleep
staging
Similar
to
sleep-wake
detection,
a
direct
relation
between
experts
agreement,
AR
coefficients
and
the
classification
performance,
was
detected
(Fig.
5).
Regarding
misclassification
of
stage
N2,
most
of
the
time,
this
stage
was
misclassified
as
stage
N1.
Moreover,
in
most
of
the
epochs
with
high
AR
coefficients
and
low
level
of
experts
agreement,
stage