Content uploaded by Stanislav Selitskiy
Author content
All content in this area was uploaded by Stanislav Selitskiy on Aug 25, 2022
Content may be subject to copyright.
192
original research
Experimental
medicine
Оригинальные исследОвания
Экспериментальная медицина
ANN–articialneuralnetworks
CNN – convolutional neural networks
EEG – electroencephalograms
LSTM – long-short term memory
ML – machine learning
MLP – multilayer perceptron
© Selitsky S., 2022
UDC 616.831-053.31:612.82
DOI – https://doi.org/10.14300/mnnc.2022.17046
ISSN – 2073-8137
S.
We propose to use a hybrid Convolutional-Multilayer Perceptron Neural Network (CNN-MLP) architecture to learn high
gamma EEG features for person recognition. An original EEG data set was collected featuring recordings during various
physical and mental activities for this study. Experiments with high and low gamma and beta scale Convolutional filters were
conducted. Particular usefulness of the EEG bands was observed, and high gamma scale CNN features were recognized as
efficient for the person recognition task.
Keywords: person recognition, EEG, high gamma, CNN-MLP, Machine Learning, Artificial Intelligent
В работе использована гибридная искусственная нейронная сеть типа сверточно-многослойного перцептрона
для изучения структур из верхнего гамма-спектра электроэнцефалографии (ЭЭГ) для распознавания людей. В свя-
зи с этим был собран исходный набор данных ЭЭГ у различных людей, включавший записи во время различных
физических действий и умственной деятельности. Выполнены исследования с высокими и низкими, гамма- и бета-
масштабными свёрточными фильтрами. В результате работы отмечалась выборочная эффективность полос ЭЭГ, а
функции сверточного многослойного персептрона с высоким гамма-масштабированием были признаны эффектив-
ными в персональном распознавании индивидуума.
Ключевые слова: ЭЭГ, персональное распознавание, высокий гамма-спектр, нейронные сети, многослойный
персептрон, машинное обучение, искусственный интеллект
For citation: Selitsky S. HYBRID CONVOLUTIONAL-MULTILAYER PERCEPTRON ARTIFICIAL NEURAL NETWORK FOR
PERSON RECOGNITION BY HIGH GAMMA EEG FEATURES. Medical News of North Caucasus. 2022;17(2):192-196. DOI –
https://doi.org/10.14300/mnnc.2022.17046
Для цитирования: Селицкий С. ГИБРИДНАЯ ИСКУССТВЕННАЯ НЕЙРОННАЯ СЕТЬ CNN-MLP ДЛЯ РАСПОЗНА-
ВАНИЯ ЛЮДЕЙ С ИСПОЛЬЗОВАНИЕМ СТРУКТУР ЧАСТОТ ВЫСОКОГО ГАММА-СПЕКТРА ЭЭГ. Медицинский вестник
Северного Кавказа. 2022;17(2):192-196. DOI – https://doi.org/10.14300/mnnc.2022.17046
A
pplication of machine learning (ML) to the ana-
lysis of electroencephalography (EEG) was
successfully applied, first, for anomaly or pa-
thology detection, such as seizure [1], depression
[2], schizophrenia [3], hearing deficiency [4], con-
sciousness level [5] detection. More recently, for
less obvious ways of human biometrics classifica-
tion, such as emotion [6–8], motor movement [9–
11], intent [12–14], sleep stages [15], newborn age
detection [16–19], and, finally, person recognition
[20] and identification [21–24]. In this pilot study,
we concentrate on person detection using EEG.
However, the data set is collected for the study,
which is still gathering data, hosts records of su-
bjects occupied with different physical, emotional,
and mental activities and suitable for various clas-
sification tasks, which are planned to be conducted
in the future. «To go to» artificial neural networks
(ANN) ML models for EEG data analysis [25–27] are
Long-short term memory (LSTM) ANN, due to their
design goal of handling sequential data of arbitrary
length.
LSTM units have saturable sigmoid gates that allow
for updating or forgetting weights of the transformation
matrix based on the history of one-by-one observation
training. The drawback of LSTM architecture is its dif-
193
2022. V 17. . 2
ficult parallelization of multiple computing processes.
If the length of the sequences in a study can be set fixed,
convolutional neural networks (CNN) are more efficient
in capturing sequence length because they can be eas-
ily trained parallelly [28–30]. Analyzing EGG records in
such fixed-length fragments using CNN and hybrid archi-
tectures is another popular ML approach [4, 9, 19, 20,
32, 36]. Input in CNN is expected as a pseudo-image, in
which the duration of the EEG record is considered as a
height of the «image,» and EEG channels are considered
either as the «width» or pseudo-color channels. The func-
tioning of the Convolution layer can be viewed as a set of
learnable filters-matrices scanning the image and being
applied to it via element-wise multiplication and summa-
tion (dot product in simple implementation).
In our study, we concentrate on using short CNN
masks designed to capture high gamma wave structures,
trying to verify the hypothesis that such waves bare useful
person-specific information that could be used for clas-
sification.
Data set. The data set is a pilot data collection from
the more considerable planned experimental study. So
far, EEG data were collected from 11 subjects during
multiple sessions with intervals of several weeks for some
subjects. The EEG sessions were of various lengths and
recorded during numerous activities, such as idleness,
lying table game, dance, motor skills acquisition, and
emotion impersonation. For EEG recording, a minimally
invasive MindRove Arc device (https://mindrove.com/
arc) was used (Fig. 1). This device is a wireless wearable
device with semi-dry electrodes and a Wi-Fi connection
to the data collection station. It has six electrodes located
at the top of the skull in the arc fashion, in the order C5,
C3, C1, C2, C4, and C6 from left to right, a bias electrode
located behind the left, and a reference electrode – be-
hind the right ear. The sampling rate is 500 Hz, and the
measurement resolution is 0.045 μV. The measurements
were done with a 50 Hz filter enabled. Although strictly
not necessary, electrode gel «Spectra 360» was used for
connectivity reliability.
Fig. 1. Mind Rove Arc with electrode enumeration.
Photo retrieved from https://mindrove.com/wp-content/
uploads/2021/11/UserManual_v2_0_0.pdf
Proposed solution. It was observed that image-
ry information generates more person-specific signa-
tures than motor-related signatures [37]. It has also
been suggested that gamma waves are associated
with visual awareness and consciousness [7]. Having
an EEG recording equipment sampling at a higher rate
(500 HZ) than equipment used in other research (circa
200–100 Hz), we decided to investigate the potentially
person-specific high gamma wave features that the CNN
filters could learn. The following (Table 1) hybrid CNN-
MLP (Multilayer Perceptron) architecture was used. In-
terest in MLP models, as rivals to CNN and using wider
than deeper architectures [31], have been rising in the
last years [33]; therefore, we utilized the best parts of
these «two worlds.» ANN parameters are filtered length
H, filter width M (equal to the number of EEG channels),
several filters N, average pooling size, and stride K. Si-
zes of the two Fully-connected layers following the Flat-
ten layer were set to match the preceding Convolutional,
Average Pooling and Flatten layers. The last Fully-con-
nected layer size was set to the number of subject clas-
ses C. The width of the convolution filters is equal to the
pseudo-image width; despite the use of 2-dimensional
Convolution layer implementation, effectively, they were
performing multi-channel 1-dimensional convolution.
The Average Pooling layer was used to reduce the dimen-
sionality of the Convolutional features input to the rest of
the fully-connected MLP layers to meet the hardware li-
mitations.
Table 1
CNN-MLP hybrid ANN architecture
Layers
Pseudo-imageInput(L×M),L–EEGfragmentlength,
M – number of channels
Pseudo2D Convolution
(H×Mlter,Nlters,1stride),H–lterheight
AveragePooling(K×1sizeandstride)
Flatten(F=(L−H+1)N/Ksize)
Fully-connected(Fsize)
ReLU
Fully-connected(2F+1size)
ReLU
Fully-connected(Csize),C–numberofclasses
Softmax
Classication
ANN parameters are filtered length H, filter width M
(equal to the number of EEG channels), several filters N,
average pooling size, and stride K. Sizes of the two Ful-
ly-connected layers following the Flatten layer were set
to match the preceding Convolutional, Average Pooling
and Flatten layers. The last Fully-connected layer size
was set to the number of subject classes C. The width
of the convolution filters is equal to the pseudo-image
width, despite the use of 2-dimensional Convolution layer
implementation; effectively, they were performing mul-
ti-channel 1-dimensional convolution. The Average Poo-
ling layer was used to reduce the dimensionality of the
Convolutional features input to the rest of the fully-con-
nected MLP layers to meet the hardware limitations.
Experiments. The experiments were run on the Li-
nux (Ubuntu 20.04.3 LTS) operating system with two dual
Tesla K80 GPUs (with 2 × 12GB GDDR5 memory each)
and one QuadroPro K6000 (with 12GB GDDR5 memo-
ry, as well), X299 chipset motherboard, 256 GB DDR4
RAM, and i9-10900X CPU. The Experiments were run on
MATLAB 2022a with Deep Learning Toolbox. ANN mo-
dels were trained using the «Adam» learning algorithm
with 0.01 initial learning coefficient, mini-batch size circa
16000, and 40 epochs. From the sets of EEG recordings
for each subject, randomly selected records for training
and records for testing. Fragments of 4 seconds duration
(or L=2000 data points) were cut from all records. The
step between cuts was 1 data point. Cuttings of the long
records were limited to 20000 first data points, thus ma-
king no more than 18001 training fragments to ensure
balanced training. No other prior processing of the train-
ing or test data was performed on the raw data. Input data
normalization, Max Pooling instead of Average Pooling,
adding Batch Normalization, and Dropout layers have
also been experimented with. In addition to the short-
194
ORIGINAL RESEARCH
Experimental
medicine
Экспериментальнаямедицина
wave CNN filter dimensions (3–13) associated with gam-
ma waves, for contrast, we ran experiments with more
extended filters associated with betta waves. Accuracy
was calculated as a ratio of correctly classified test data
points to the whole number of test data points.
Results and Discussion. Initial min-max per-channel
data normalization applied to each 4-seconds EEG
fragment resulted in a random accuracy of circa 0.09;
therefore, all subsequent experiments were run without
normalization. ANN architecture used for experiments
did not have saturable sigmoid activation functions,
and input channels had the same measurement unit
data; therefore, input data normalization was not strictly
needed. Adding Batch Normalization and Dropout layers
to first test architectures has smoothed the training
process; however, it produced marginally worse accuracy
results and slowed down training. Therefore, those layers
were not used for the majority of experiments. Using the
Max Pooling layer instead of Average Pooling in the first
runs produced worse results and was retired for further
investigations. The pooling size and stride parameters K
of 2 and 5 were experimented with and provided inferior
results to the parameter value of 3, or training did not
even converge for the former parameters. Therefore,
subsequent experiments were conducted with the latter
parameter of 3. The primary Convolutional filter height
parameter H=5 corresponds to wave features at 100 Hz,
as compared with other values (Table 2).
Table 2
Accuracy for various Convolution layer filter height
(width is set to number of channels 6), number
of filters 16, and pooling size and stride 3
CNNltersize Frequency [Hz] Accuracy
3 167 0.3478
5 100 0.4752
7 71 0.1870
13 38 0.4721
17 29 0.3442
31 16 0.4752
37 13 0.3478
The number of filters N was pushed to the maximal
value allowed by the GPU for the given ANN architec-
ture – of 16. Examples of the filters learnt for H=5, K=3,
N=16 can be seen on Figure 2, and for the beta frequency
filter height H=31 – on Figure 3. The latter, longer CNN
filters capture repetition of the higher frequency features.
Fig. 2. Convolution filter examples,
16 filters of height 5 and 6 channels
Fig. 3. Convolution filter examples,
16 filters of height 31 and 6 channels
Examples of the activation values on the Convolution
layer output, which could be viewed as intensity maps
of the features represented by the learned filters, can
be seen for correctly identified test example (data point
140 000) in Figure 4 and for incorrectly identified test
example (data point 120 000) on Figure 5. The latter case
likely demonstrates recording artifacts.
Fig. 4. Convolution layer activation values for each
of 16 5×6 filters. Correctly classified data point 140 000
Fig. 5. Convolution layer activation values for each
of 16 5×6 filters. Incorrectly classified data point 120 000
195
MEDICAL NEWS OF NORTH CAUCASUS
2022. Vоl. 17. Iss. 2
МЕДИЦИНСКИЙ ВЕСТНИК СЕВЕРНОГО КАВКАЗА
2022. Т. 17. № 2
In such a way, areas in the image similar to the filter mask
will produce a higher output than non-similar areas. The fil-
ters are randomly initialized and trained via standard back-
propagation learning algorithms to create masks that would
minimize the objective classification function. Another ANN
architecture proven to be effective in the case of the small
EGG and other medical imaging sets is evolutionary polyno-
mial GMDH-type (Group Method of Data Handling) [28, 23,
38]. Yet another popular ML branch of Decision Trees (DT)
has also been applied to EEG problems [13, 25, 29].
Conclusions. High frequency «hyper gamma» EEG
features can be detected by the CNN architecture and
successfully used by the following MLP layers for person
identification. There are closely intertwined frequency
bands (circa 100, 40, 15 Hz) that are more suitable and
less suitable (circa 170, 70, 40, 13 Hz) for personal iden-
tification. Features learned by the Convolution filters sized
for lower frequencies do not represent unique structures
but rather a repetition of higher frequency features, which
indicates the latter’s importance for person recognition.
Parallel Convolution cascade cells covering ranges of
frequency bands, which should be capable of capturing
more complex wave structures, would be a natural area
for further research. Activation outputs cluster together
for some filters, indicating redundancy and the potential
to reduce them to free resources for more complex cas-
cade architectures. Cases of inaccurate classification in
the proposed solutions may be associated with unfiltered
artifacts; therefore, more tight data pre-processing would
be another area of methodology improvement.
Disclosures: The authors declare no conflict of interest.
Acknowledgments. The authors are thankful to Dr. L. Jakaite and Dr. V. Schetinin for the research inspiration and
provided support.
1. Acharya U. R., Oh S. L., Hagiwara Y., Tan J. H., Adeli H.
Deep convolutional neural network for the automated
detection and diagnosis of seizure using EEG signals.
Computers in biology and medicine. 2018;100:270-278.
https://doi.org/10.1016/j.compbiomed.2017.09.017
2. Greco C., Matarazzo O., Cordasco G., Vinciarelli A.,
Callejas Z. [et al.] Discriminative power of EEG-based
biomarkers in major depressive disorder: A systematic
review. IEEE Access. 2021;РР:1-1.
https://doi.org/10.1109/ACCESS.2021.3103047
3. Khare S. K., Bajaj V., Siuly S., Sinha G. Classication
of schizophrenia pa- tients through empirical wavelet
transformation using electroencephalogram signals.
Modelling and Analysis of Active Biopotential Signals in
Healthcare. 2020;1:1-26.
https://doi.org/10.1088/978-0-7503-3279-8ch1
4.
Islam M. N., Sulaiman N., Al Farid F., Uddin J., Alyami S. A.
[et al.] Diagnosis of hearing deciency using EEG based
AEP signals: Cwt and improved-vgg16 pipeline. Peer J.
Computer Science. 2021;7,e638.
https://doi.org/10.7717/peerj-cs.638
5. Liu Q., Cai J., Fan S. Z., Abbod M. F., Shieh J. S. [et al.]
Spectrum analysis of EEG signals using cnn to model
patient’s consciousness level based on anesthesiologists’
experience. IEEE. 2019;7:53731-53742.
https://doi.org/10.1109/ACCESS.2019.2912273
6. Alhagry S., Fahmy A. A., El-Khoribi R. A. Emotion
recognition based on eeg using lstm recurrent neural
network. Emotion. 2017;8(10):355-358.
7. Du X., Ma C., Zhang G., Li J., Lai Y. K. [et al.] An ecient
lSTM network for emotion recognition from multichannel
EEG signals. IEEE Trans. Aect. Comput. 2020.
https://doi.org/10.1109/TAFFC.2020.3013711
8. Xing X., Li Z., Xu T., Shu L., Hu B. [et al.] SAE+ LSTM:
A new framework for emotion recognition from multi-
channel EEG. Frontiers in neurorobotics. 2019;13(37):1-14.
https://doi.org/10.3389/fnbot.2019.00037
9. Sheykhivand S., Mousavi Z., Rezaii T. Y., Farzamnia A.
Recognizing emotions evoked by music using cnn-lstm
networks on EEG signals. IEEE. 2020:139332-139345
https://doi.org/10.1088/1742-6596/2024/1/012044
10. Alarcao S. M., Fonseca M. J. Emotions recognition using
EEG signals: A survey. IEEE Transactions on Aective
Computing. 2017;10(3):374-393.
https://doi.org/10.1109/TAFFC.2017.2714671
11. Zhang G., Davoodnia V., Sepas-Moghaddam A., Zhang
Y., Etemad A. Classication of hand movements from EEG
using a deep attention-based lstm network. IEEE Sensors
Journal. 2019;20(6):3113-3122.
https://doi.org/10.1109/JSEN.2019.2956998
12. Zhou H., Zhao X., Zhang H., Kuang S. The mechanism
of a multi-branch structure for EEG-based motor imagery
classication. In: 2019 IEEE International Conference on
Robotics and Biomimetics (ROBIO). IEEE. 2019:2473-
2477. https://doi.org/10.3390/diagnostics12040995
13. Kurkin S. A., Pitsik E. N., Musatov V. Y., Runnova A. E.,
Hramov A. E. Articial neural networks as a tool for
recognition of movements by electroencephalograms.
In: ICINCO. 2018;1:176-181. Available at: https://www.
scitepress.org/papers/2018/68602/68602.pdf
14. Vega C. F., Quevedo J., Escandón E., Kiani M., Ding W.
[et al.] Fuzzy temporal convolutional neural networks in
p300-based brain – computer interface for smart home
interaction. Applied Soft Computing. 2022;117:108359.
https://doi.org/10.1016/j.asoc.2021.108359
15. Kang J. S., Park U., Gonuguntla V., Veluvolu K. C., Lee M.
Human implicit intent recognition based on the phase
synchrony of EEG signals. Pattern Recognition Letters.
2015;66:144-152.
https://doi.org/10.1016/j.patrec.2015.06.013
16. Schetinin V., Jakaite L., Schult J. Informativeness of
sleep cycle features in bayesian assessment of newborn
electroencephalographic maturation. DBLP. 2011:1-6.
https://doi.org/10.1109/CBMS.2011.5999111
17. Jakaite L., Schetinin V., Maple C. Bayesian assessment
of newborn brain maturity from two-channel sleep
electroencephalograms. Computational and Mathematical
Methods in Medicine. 2012:629654.
https://doi.org/10.1155/2012/629654
18. Schetinin V., Jakaite L. Classication of newborn EEG
maturity with Bayesian averaging over decision trees.
Expert Systems with Applications. 2012;39(10):9340-9347.
https://doi.org/10.1016/j.eswa.2012.02.184
19. Selitsky S., Selitskaya N., Schult J. Machine learning
approach to classication of sleep electroencephalograms
from newborns at risk of brain pathologies. Medical News
of North Caucasus. 2021;16(2):140-143.
https://doi.org/10.14300/mnnc.2021.16031
20. Schetinin V., Jakaite L., Nyah N., Novakovic D., Krzanows-
ki W. Feature extraction with GMDH-type neural networks
for EEG-based person identication. International Journal
of Neural Systems. 2018;28(06):1750064.
https://doi.org/10.1142/S0129065717500642
21. Jakaite L., Schetinin V., Schult J. Feature extraction
from electroencephalograms for Bayesian assessment of
newborn brain maturity. In: Proceedings of the 24th IEEE
International Symposium on Computer-Based Medical
Systems. 2011
https://doi.org/10.1109/CBMS.2011.5999109
22. La Rocca D., Campisi P., Scarano G. EEG biometrics for
individual recognition in resting state with closed eyes. In:
2012 BIOSIG-Proceedings of the International Conference
of Biometrics Special Interest Group (BIOSIG). IEEE.
2012:1-12.
23. Hammad M., Pławiak P., Wang K., Acharya U. R. Resnet-
attention model for human authentication using EEG
signals. Expert Systems. 2021;38(6):e12547.
https://doi.org/10.1111/exsy.12547
24. Michielli N., Acharya U. R., Molinari F. Cascaded
lstm recurrent neural network for automated sleep
stage classication using single-channel EEG signals.
196
Оригинальные исследОвания
Морфология
original research
Morphology
Computers in biology and medicine. 2019;106:71-81.
https://doi.org/10.1016/j.compbiomed.2019.01.013
25. Lawhern V. J., Solon A. J., Waytowich N. R., Gordon S. M.,
Hung C. P., Lance B. J. EEGnet: a compact convolutional
neural network for EEG-based brain – computer interfaces.
Journal of Neural Engineering. 2018;15(5):056013.
https://doi.org/10.1088/1741-2552/aace8c
26. Amin S. U., Muhammad G., Abdul W., Bencherif M.,
Alsulaiman M. Multi-CNN feature fusion for ecient EEG
classication. In: 2020 IEEE International Conference on
Multimedia & Expo Workshops (ICMEW). 2020:1-6.
https://doi.org/10.1109/icmew46912.2020.9106021
27. Sheykhivand S., Mousavi Z., Rezaii T. Y., Farzamnia A.
Recognizing emotions evoked by music using CNN-LSTM
networks on EEG signals. IEEE. 2020:139332-139345.
https://doi.org/10.3390/s22062346
28. Schetinin V., Schult J. Learning polynomial networks for
classication of clinical electroencephalograms. Soft
Computing. 2006;10(4):397-403.
https://doi.org/10.1007/s00500-005-0499-3
29. Nyah N., Jakaite L., Schetinin V., Sant P., Aggoun A.
Evolving polynomial neural networks for detecting
abnormal patterns. 8th IEEE International Conference on
Intelligent Systems. IEEE. 2016:74-80.
https://doi.org/10.1109/IS.2016.7737403
30. Nguyen P., Tran D., Huang X., Sharma D. A proposed
feature extraction method for EEG-based person identi-
cation. In: Proceedings on the International Conference on
Articial Intelligence. 2012.
31. Chang W., Wang H., Yan G., Liu C. An EEG based familiar
and unfamiliar person identication and classication
system using feature extraction and directed functional
brain network. Expert Systems with Applications.
2020:113448. https://doi.org/10.3390/brainsci11111424
32. Gold I. Does 40-hz oscillation play a role in visual
consciousness? Consciousness and Cognition. 1999;8(2):
186-195. Available at: https://www.sciencedirect.com/
science/article/pii/S1053810099903999
33. Nguyen P., Tran D., Huang X., Sharma D. A proposed
feature extraction method for EEG-based person
identication. In: Proceedings on the International
Conference on Articial Intelligence (ICAI). 2012.
34. Schetinin V., Jakaite L. Classication of newborn EEG
maturity with bayesian averaging over decision trees.
Expert Syst. 2012;39(10):9340-9347.
https://doi.org/10.1016/j.eswa.2012.02.184
35. Selitskiy S. Kolmogorov’s gate non-linearity as a step
toward much smaller articial neural networks. In:
Proceedings of the 24th International Conference on
Enterprise Information Systems. ICEIS. 2022:492-499.
https://doi.org/10.5220/0011060700003179
36. Tolstikhin I. O., Houlsby N., Kolesnikov A., Beyer L., Zhai X.
[et al.] MLP-mixer: An all-MLP architecture for vision.
Advances in Neural Information Processing Systems.
2021. Available at: https://papers.nips.cc/paper/2021/le/
cba0a4ee5ccd02fda0fe3f9a3e7b89fe-Paper.pdf
37. Yang H., Han J., Min K. A multi-column CNN model
for emotion recognition from EEG signals. Sensors.
2019;19(21):4736. https://doi.org/10.3390/s19214736
38. Yang S., Deravi F. On the eectiveness of EEG signals as a
source of biometric information. In: 2012 Third International
Conference on Emerging Security Technologies. IEEE.
2012:49-52. https://doi.org/10.1109/EST.2012.8
About author
Selitsky Stas, postgraduate student at the School of Computer Science and Technology; e-mail: selitsky@yahoo.com
© Group of authors, 2022
UDC 611.127.13.142
DOI – https://doi.org/10.14300/mnnc.2022.17047
ISSN – 2073-8137
The anterior intraventricular branch has been examined in the systole phase on 60 antemortem angiograms with
left-anterior and right-vertebral variants of branching of coronal arteries in people of second adult age with a myocardial
bridge. Data on a comparative examination of the lumen of the anterior intraventricular branch at the left-vertebral and
right-vertebral variants of branching of coronal arteries in the subepicardial and intramural departments of the main high-
way are presented.
Keywords: myocardial bridge, anterior interventricular branch, systole, left variant of coronary branching, right variant of
coronary branching, internal diameter