PreprintPDF Available

Brain-Computer-Interface controlled robot via RaspberryPi and PiEEG. arXiv:2202.01936

Authors:
Ildar Rakhmatulin at Heriot-Watt University
Ildar Rakhmatulin
Sebastian Völkl at hackerBCI
Sebastian Völkl
  • hackerBCI
Preprints and early-stage research may not have been peer reviewed yet.
Download file PDF
Read file
Download citation

Abstract and Figures

This paper presents Open-source software and a developed shield board for the Raspberry Pi family of single-board computers that can be used to read EEG signals. We have described the mechanism for reading EEG signals and decomposing them into a Fourier series and provided examples of controlling LEDs and a toy robot by blinking. Finally, we discussed the prospects of the brain-computer interface for the near future and considered various methods for controlling external mechanical objects using real-time EEG signals. Links Source-https://github.com/Ildaron/EEGwithRaspberryPI/tree/master/Robot_control
Figures - uploaded by Ildar Rakhmatulin
Author content
All figure content in this area was uploaded by Ildar Rakhmatulin
Content may be subject to copyright.
Content uploaded by Ildar Rakhmatulin
Author content
All content in this area was uploaded by Ildar Rakhmatulin on Feb 07, 2022
Content may be subject to copyright.
Brain-Computer-Interface controlled robot via RaspberryPi and PiEEG.
Ildar Rakhmatulin* – Ph.D. Electronic Researcher
Sebastian Völkl - Brain-Computer-Interface Developer
Abstract
This paper presents Open-source software and a developed shield board for the Raspberry Pi family of
single-board computers that can be used to read EEG signals. We have described the mechanism for reading
EEG signals and decomposing them into a Fourier series and provided examples of controlling LEDs and
a toy robot by blinking. Finally, we discussed the prospects of the brain-computer interface for the near
future and considered various methods for controlling external mechanical objects using real-time EEG
signals.
Links
Source - https://github.com/Ildaron/EEGwithRaspberryPI/tree/master/Robot_control
Web-Site - https://www.hackerbci.com/
YouTube – https://youtu.be/wNgCEKIXGUY
Slack - pieeg.slack.com
*email: ildarr2016@gmail.com
License - GNU General Public License v3.0
Keywords: PIEEG, hackerbci, RaspberryPi, EEG, brain-computer interface
Abbreviation
BCI Brain-computer interface
EEG Electroencephalogram
SBC Single-board computer
ADC Analog-digital converter
Introduction
When the term BCI is mentioned, many people immediately associate with controlling objects using the
power of thought. Now, neuroscience in non-invasive EEG measurement is only getting there. Still, each
step brings us closer to that goal and inspires a new generation of scientists and engineers to contribute to
this field of science. We have machine learning, which entered our lives only a few years ago, and more
than enough computing power to look for correlations in EEG signals. The only weakness will be the
availability of the dataset. Therefore, we hope that the existence of a device at a low price will allow us to
take a step towards solving this problem.
Reading EEG signals, despite its apparent simplicity - measuring microvolts from the scalp through
electrodes with a high-precision ADC - is associated with various scientific fields. It involves reading EEG
signals [1, 2022], processing EEG signals [2, 2021], selecting features, and finally using the signals for
various purposes. Moreover, blinks or chews are unwanted artifacts that introduce harmful distortions into
EEG signals, and many works are devoted to combating these artifacts [3, 2022; 4, 2022]. At the same time,
however, these artifacts are still commonly used in applied tasks, such as blink control of external objects.
Lin et al. [5, 2010] successfully controlled an electric wheelchair by blinking through a brain-computer
interface. Huang et al. [6, 2019] developed an application to control the integrated system of a robotic
wheelchair by blinking and BCI.
Our board is designed to familiarize everyone with the world of EEG, including those not directly related
to the field of neurology. So, our mission is to lower the threshold of technical knowledge to get started
with BCI. Our goal was not to compete with the previously described papers and show that our control
method has the best performance. Considering all the shortcomings of this method (competition with eye-
tracking, low efficiency, limited freedom of action, and complexity in administration), we believe this
method is an excellent example to demonstrate the capabilities of BCIs.
1. Technical details
The technical points of the developed device PIEEG were presented in the article [7, 2022]. Therefore, in
this paper, we only limit ourselves to the main technical details of the device in table 1.
Table 1. PIEEG characteristic
Fig. 1 schematically shows the use of the developed device for robotic objects control.
a b
Fig. 1.a - Schematic representation of the use of PIEEG, b -8 electrodes connected by according to the
International 10-20 system
We used dry electrodes from Florida research instruments (https://fri-fl-shop.com/collections/electrodes-
eeg-electrodes). An overview of dry electrodes, their uses, and disadvantages we have described in the
paper [10, 2021].
2. Software
An overview of software for processing EEG signals was given earlier in this paper [8, 2021]. The main
motive for developing our own software rather than integrating it into existing platforms is the expediency
of constantly changing and improving the software, for example, for mind control of the robot. We have
made the connection of RaspberryPi and ADC in C language and created a static library and implemented
the subsequent signal processing in Python. The static library sets the signaling rate and registers for setting
up the ADC. For us, it is essential that when a Python file is corrected, the library does not need to be
recompiled. Figure 2 shows the process of measuring EEG signals and blink artifacts.
Fig.2. Blinking with EEG signal on PIEEG (real-time)
As shown in the figure, the response in the EEG signals to the blink is available on all channels, so in the
research, it is sufficient for us to use only one electrode; we chose the electrode in position - Fz.
3.1 Device setup
The position of the electrode can be changed and adjusted for both chewing detection and blinking. For the
measurement process, the PIEEG should be connected to the Raspberry PI3 or RaspberryPI4. The power
supply for the RaspberryPi must be connected to a battery - 5V (do not use the power network).
The method we chose for feature selection is the Fast Fourier Transform, where the signal is decomposed
into frequencies [9, 2021]. Repetitive actions such as blinking or chewing. It is worth noting that blinking
at a specific frequency is quite tricky, which is done intuitively. The signal amplitude is different for each
person (the response of EEG signals to blinking), so the settings for the frequency range of the response
and the amplitude of the signal are found by each user through experimental methods. In Figs. 3 and 4, we
show the results of controlling two LEDs by blinking. We found amplitude peaks in the frequency ranges
of 5 Hz and 3 Hz for the EEG signal decomposed by Fast Fourier Transform.
a
b
Fig.3. LED control in the range of 3 - 7 Hz. a (upper graph) - EEG signal through pass filter 1-30 Hz. an
(upper c) – EEG signal (the part circled in purple from the graph above) received through the Fast Fourier
Transform. The red line is the setting for turning on the LED, green lines are the frequency range in which
we monitor the signal amplitude. b – LED circled in blue, output - 31 for GPIO.setmode (GPIO.BOARD)
a
b
Fig.4. LED control in the range of 1 - 3 Hz. a (upper graph) - EEG signal through pass filter 1-30 Hz. a
(lower upper graph) – EEG signal (the part circled in purple from the graph above) received through the
Fast Fourier Transform. Redline is the setting for turning on the LED, green lines are the frequency range
in which we monitor the signal amplitude. b – LED circled in blue, output - 35 for GPIO.setmode
(GPIO.BOARD)
The result is positive but does not look ideal. The accuracy can be improved by setting other ranges for the
bandpass filter, changing the location of the electrode, changing ADC registers, et cetera. In our case, the
instrument worked correctly in 8 out of 10 cases for both frequency ranges. This error can be caused by the
hardware characteristics of the instrument, the software, and human factors. For better illustration, we did
not connect the signals to the LED but the remote control of the radio-controlled toy and were able to
successfully control the toy’s movement (Fig. 5).
Fig.5. Connecting toys. Discrete signals are connected to the control buttons of the radio remote control.
Circled in red and green squares.
The toy is presented only as an example, any type of device can be connected.
Conclusion
We suspect that controlling robots by thought would ideally have more to do with motor imagery. However,
that has not been done with non-invasive electrodes yet. All the studies we have analyzed generally work
in two coordinates - x and y, and the z coordinate. Therefore, the ease of implementation of this method
will hopefully inspire engineers to move forward in this direction. We still believe that non-invasive
electrodes will soon achieve better results, thanks to the power of machine learning, allowing more natural
control of robots.
We did not want to show that blinking is the best way to control a robot, but we wanted to use cheap devices
to demonstrate how to control external objects using EEG signals. Paradigms such as P300 and ERPs that
allow printing on the keyboard can also be used in similar scenarios.
Now the software works in real time with a delay of 1 second, if necessary, the reaction speed can be
increased. One of the advantages of working with a shield is that the data from the ADC is transferred
directly to the processor using the SPI protocol. We do not need to use adapters, transmitters, such as
Bluetooth, Wi-Fi, Serial. Ideally, when implementing software in a lower programming language, the
response speed allows us to receive data at the transfer rate set in the ADS1299 - 250 SPS to 16 kSPS.
The shield can be installed on the OrangePi and BananaPi series SBCs. It is worth noting that the connection
to the boards is made through the power supply, SPI channels and several discrete outputs, so the devices
can be connected to any SBCs through wires if the GPIO channels do not match. All the software is in the
public domain and is under an open license, all rights and comments are welcome. We will continue to
work on the software and the shield in the future.
References
1. Yu, H,. Ba, S,. Guo, Y., Guo, L., Xu, G. (2022). Effects of Motor Imagery Tasks on Brain Functional
Networks Based on EEG Mu/Beta Rhythm.Brain Sci.,12, 194. https://doi.org/10.3390/brainsci12020194
2. Luján, M., Jimeno, M., Mateo, S.,et al. (2021). A Survey on EEG Signal Processing Techniques and
Machine Learning: Applications to the Neurofeedback of Autobiographical Memory Deficits in
Schizophrenia. Electronics, 10, 3037. https://doi.org/10.3390/electronics10233037
3. McDermott, E.J.; Raggam, P. (2022). Artifacts in EEG-Based BCI Therapies: Friend or Foe? Sensor ,22,
96. https://doi.org/10.3390/s22010096
4. Maddirala, K.; Veluvolu, C. (2022). SSA with CWT and k-Means for Eye-Blink Artifact Removal from
Single-Channel EEG Signals. Sensors,22, 931. https://doi.org/10.3390/s22030931
5. Lin, J., Chen, K., Yang, W. (2010). EEG and eye-blinking signals through a Brain-Computer Interface
based control for electric wheelchairs with wireless scheme. 4th International Conference on New Trends
in Information Science and Service Science, 731-734.
6. Huang, Q., Zhang, Z., Yu, T., He, S., & Li, Y. (2019). An EEG-/EOG-Based Hybrid Brain-Computer
Interface: Application on Controlling an Integrated Wheelchair Robotic Arm System. Frontiers in
neuroscience, 13, 1243. https://doi.org/10.3389/fnins.2019.01243
7. Rakhmatulin, I., Volkl, S. (2022). PIEEG: Turn a Raspberry Pi into a Brain-Computer-Interface to
measure biosignals, arXiv:2201.02228
8. Rakhmatulin, I., Parfenov, A., Traylor, Z. et al. Low-cost brain computer interface for everyday use. Exp
Brain Res 239, 3573–3583 (2021). https://doi.org/10.1007/s00221-021-06231-4
9. Rakhmatulin, I. Review of EEG Feature Selection by Neural Networks. Int. J. Sci. Bus. 2020, 4, 101–
112
10. Rakhmatulin, I., & Gan, Y. (2021). Review Dry and Non-Contact EEG Electrodes for 2010-2021 Years.
Preprints, 2021030555. https://doi:10.20944/preprints202103.0555.v1
ResearchGate has not been able to resolve any citations for this publication.
Effects of Motor Imagery Tasks on Brain Functional Networks Based on EEG Mu/Beta Rhythm
Article
Full-text available
Motor imagery (MI) refers to the mental rehearsal of movement in the absence of overt motor action, which can activate or inhibit cortical excitability. EEG mu/beta oscillations recorded over the human motor cortex have been shown to be consistently suppressed during both the imagination and performance of movements, although the specific effect on brain function remains to be confirmed. In this study, Granger causality (GC) was used to construct the brain functional network of subjects during motor imagery and resting state based on EEG in order to explore the effects of motor imagery on brain function. Parameters of the brain functional network were compared and analyzed, including degree, clustering coefficient, characteristic path length and global efficiency of EEG mu/beta rhythm in different states. The results showed that the clustering coefficient and efficiency of EEG mu/beta rhythm decreased significantly during motor imagery (p < 0.05), while degree distribution and characteristic path length increased significantly (p < 0.05), mainly concentrated in the frontal lobe and sensorimotor area. For the resting state after motor imagery, the changes of brain functional characteristics were roughly similar to those of the task state. Therefore, it is concluded that motor imagery plays an important role in activation of cortical excitability.
View
SSA with CWT and k-Means for Eye-Blink Artifact Removal from Single-Channel EEG Signals
Article
Full-text available
Recently, the use of portable electroencephalogram (EEG) devices to record brain signals in both health care monitoring and in other applications, such as fatigue detection in drivers, has been increased due to its low cost and ease of use. However, the measured EEG signals always mix with the electrooculogram (EOG), which are results due to eyelid blinking or eye movements. The eye-blinking/movement is an uncontrollable activity that results in a high-amplitude slow-time varying component that is mixed in the measured EEG signal. The presence of these artifacts misled our understanding of the underlying brain state. As the portable EEG devices comprise few EEG channels or sometimes a single EEG channel, classical artifact removal techniques such as blind source separation methods cannot be used to remove these artifacts from a single-channel EEG signal. Hence, there is a demand for the development of new single-channel-based artifact removal techniques. Singular spectrum analysis (SSA) has been widely used as a single-channel-based eye-blink artifact removal technique. However, while removing the artifact, the low-frequency components from the non-artifact region of the EEG signal are also removed by SSA. To preserve these low-frequency components, in this paper, we have proposed a new methodology by integrating the SSA with continuous wavelet transform (CWT) and the k-means clustering algorithm that removes the eye-blink artifact from the single-channel EEG signals without altering the low frequencies of the EEG signal. The proposed method is evaluated on both synthetic and real EEG signals. The results also show the superiority of the proposed method over the existing methods.
View
Artifacts in EEG-Based BCI Therapies: Friend or Foe?
Article
Full-text available
EEG-based brain–computer interfaces (BCI) have promising therapeutic potential beyond traditional neurofeedback training, such as enabling personalized and optimized virtual reality (VR) neurorehabilitation paradigms where the timing and parameters of the visual experience is synchronized with specific brain states. While BCI algorithms are often designed to focus on whichever portion of a signal is most informative, in these brain-state-synchronized applications, it is of critical importance that the resulting decoder is sensitive to physiological brain activity representative of various mental states, and not to artifacts, such as those arising from naturalistic movements. In this study, we compare the relative classification accuracy with which different motor tasks can be decoded from both extracted brain activity and artifacts contained in the EEG signal. EEG data were collected from 17 chronic stroke patients while performing six different head, hand, and arm movements in a realistic VR-based neurorehabilitation paradigm. Results show that the artifactual component of the EEG signal is significantly more informative than brain activity with respect to classification accuracy. This finding is consistent across different feature extraction methods and classification pipelines. While informative brain signals can be recovered with suitable cleaning procedures, we recommend that features should not be designed solely to maximize classification accuracy, as this could select for remaining artifactual components. We also propose the use of machine learning approaches that are interpretable to verify that classification is driven by physiological brain states. In summary, whereas informative artifacts are a helpful friend in BCI-based communication applications, they can be a problematic foe in the estimation of physiological brain states.
View
A Survey on EEG Signal Processing Techniques and Machine Learning: Applications to the Neurofeedback of Autobiographical Memory Deficits in Schizophrenia
Article
Full-text available
  • Dec 2021
In this paper, a general overview regarding neural recording, classical signal processing techniques and machine learning classification algorithms applied to monitor brain activity is presented. Currently, several approaches classified as electrical, magnetic, neuroimaging recordings and brain stimulations are available to obtain neural activity of the human brain. Among them, non-invasive methods like electroencephalography (EEG) are commonly employed, as they can provide a high degree of temporal resolution (on the order of milliseconds) and acceptable space resolution. In addition, it is simple, quick, and does not create any physical harm or stress to patients. Concerning signal processing, once the neural signals are acquired, different procedures can be applied for feature extraction. In particular, brain signals are normally processed in time, frequency, and/or space domains. The features extracted are then used for signal classification depending on its characteristics such us the mean, variance or band power. The role of machine learning in this regard has become of key importance during the last years due to its high capacity to analyze complex amounts of data. The algorithms employed are generally classified in supervised, unsupervised and reinforcement techniques. A deep review of the most used machine learning algorithms and the advantages/drawbacks of most used methods is presented. Finally, a study of these procedures utilized in a very specific and novel research field of electroencephalography, i.e., autobiographical memory deficits in schizophrenia, is outlined.
View
Low-cost brain computer interface for everyday use
Article
Full-text available
  • Dec 2021
  • EXP BRAIN RES
With the growth in electroencephalography (EEG) based applications the demand for affordable consumer solutions is increasing. Here we describe a compact, low-cost EEG device suitable for daily use. The data are transferred from the device to a personal server using the TCP-IP protocol, allowing for wireless operation and a decent range of motion for the user. The device is compact, having a circular shape with a radius of only 25 mm, which would allow for comfortable daily use during both daytime and nighttime. Our solution is also very cost effective, approximately $350 for 24 electrodes. The built-in noise suppression capability improves the accuracy of recordings with a peak input noise below 0.35 μV. Here, we provide the results of the tests for the developed device. On our GitHub page, we provide detailed specification of the steps involved in building this EEG device which should be helpful to readers designing similar devices for their needs https://github.com/Ildaron/ironbci.
View
Review Dry and Non-contact EEG Electrodes for 2010-2021 years
Preprint
Full-text available
  • Mar 2021
The basis of the work of electroencephalography (EEG) is the registration of electrical impulses from the brain or some of its individual areas using a special sensor/electrode. This method is used for the treatment and diagnosis of various diseases. The use of wet electrodes in this case does not seem viable, for several well-known reasons. As a result of this, a detailed analysis of modern EEG sensors developed over the past few years is carried out, which will allow researchers to choose this type of sensor more carefully and, as a result, conduct their research more competently. Due to the absence of any standards in the production and testing of dry EEG sensors, the main moment of this manuscript is a detailed description of the necessary steps for testing a dry electrode, which will allow researchers to maximize the potential of the sensor in the various type of research. (PDF) Review Dry and Non-contact EEG Electrodes for 2010-2021 years. Available from: https://www.researchgate.net/publication/350313034_Review_Dry_and_Non-contact_EEG_Electrodes_for_2010-2021_years [accessed Mar 23 2021].
View
Review of EEG feature selection by neural networks
Article
Full-text available
  • Aug 2020
The basis of the work of electroencephalography (EEG) is the registration of electrical impulses from the brain using a special sensor or electrode. This method is used to treat and diagnose various diseases. In the past few years, due to the development of neural network technologies, the interest of researchers in EEG has noticeably increased. Neural networks for training the model require obtaining data with minimal noise distortion. In the processing of EEG signals to eliminate noise (artifacts), signal filtering and various methods for extracting signs are used. The presented manuscript provides a detailed analysis of modern methods for extracting the signs of an EEG signal used in studies of the last decade. The information presented in this paper will allow researchers to understand how to more carefully process the data of EEG signals before using neural networks to classify the signal. Due to the absence of any standards in the method of extracting EEG signs, the most important moment of this manuscript is a detailed description of the necessary steps for recognizing artifacts, which will allow researchers to maximize the potential of neural networks in the tasks of classifying EEG signal. IJSB Introduction Feature extraction is the process of extracting useful information from an electroencephalogram (EEG) signal to represent the correct data set before performing the classification procedure. Over the past decade, the database of the US National Library of Medicine, the popular libraries of Elsevier, Taylor & Francis, ieeexplore.ieee.org, more than 500 works dedicated to the allocation of processing of EEG signals were presented. The analysis of these works allows us to conclude that the surge of interest in this topic is since it became possible to use various mxodels of neural networks as a classification tool. These researches focus on the signal EEG classification stage. The choice of the electrode does not carry great difficulties, due to the simplicity of the criterion for its selection (impedance, noise, etc.), but the competent implementation of the signs carry certain difficulties. For models of neural networks engaged in the tasks of signal classification, the main criterion for the correct operation is the correct receipt of a dataset. Therefore, the purpose of this manuscript is to conduct a detailed analysis of modern methods for extracting features of EEG signals. This study is especially relevant when identifying features for use in classification problems using machine learning. The following methods for extracting features were considered by Al-Fahoum (2014): temporal frequency distributions (TFD), fast Fourier transform (FFT), eigenvector methods (EM), wavelet transform (WT) and autoregressive method (ARM). In the paper, the characteristics of various methods were briefly compared. Mane et al. (2019), similarly, as and author in the article described above, various methods of feature extraction are compared, such as wavelet transform, independent component analysis, principal component analysis, autoregressive model, and empirical mode decomposition. Raut (2010) in his paper considered three subsets of the obtained characteristics by the method of extracting tracks, wavelet transform, and fractional Fourier transform. The author carried a comparison of efficiency out by classification using reference vector machines. Despite the stated goals, the described works did not fully reveal the potential and significance of the feature extraction technique in classification problems by machine learning. The main disadvantage:-these works describe several methods for extracting attributes and testing them on one of the models of neural networks, that, according to the author, does not yet prove the correctness of the operation for extracting features. Since it is necessary to use several neural networks for verification;-the recommendations are not presented in the works on the correct extraction of features in the tasks of classifying EEG signals for subsequent studies. This manuscript will try to avoid these shortcomings and fully disclose a method for extracting features from EEG signals. The algorithm for interpreting EEG signals consists of 4 steps, figure 1.
View
An EEG-/EOG-Based Hybrid Brain-Computer Interface: Application on Controlling an Integrated Wheelchair Robotic Arm System
Article
Full-text available
  • Nov 2019
Most existing brain-computer Interfaces (BCIs) are designed to control a single assistive device, such as a wheelchair, a robotic arm or a prosthetic limb. However, many daily tasks require combined functions which can only be realized by integrating multiple robotic devices. Such integration raises the requirement of the control accuracy and is more challenging to achieve a reliable control compared with the single device case. In this study, we propose a novel hybrid BCI with high accuracy based on electroencephalogram (EEG) and electrooculogram (EOG) to control an integrated wheelchair robotic arm system. The user turns the wheelchair left/right by performing left/right hand motor imagery (MI), and generates other commands for the wheelchair and the robotic arm by performing eye blinks and eyebrow raising movements. Twenty-two subjects participated in a MI training session and five of them completed a mobile self-drinking experiment, which was designed purposely with high accuracy requirements. The results demonstrated that the proposed hBCI could provide satisfied control accuracy for a system that consists of multiple robotic devices, and showed the potential of BCI-controlled systems to be applied in complex daily tasks.
View
EEG and eye-blinking signals through a Brain-Computer Interface based control for electric wheelchairs with wireless scheme
Conference Paper
Full-text available
  • Jun 2010
This paper mainly use simple unipolar electrode to capture EEG from the forehead to build a Brain-Computer Interface (BCI) based control for electric wheelchairs through Bluetooth for paralyzed patients. We have normalized β, α, θ and δ waves to construct two signals such as meditation and attention. Additionally, we can also extract the eye-blinking signals from BCI. Therefore, attention and eye-blinking signals can be collected as the control signals through a Bluetooth interface and the electrical interface in electric wheelchair. The experimental results confirmed that this system can provide a convenient manner to control an electric wheelchair.
View
View