The assistive, adaptive, and rehabilitative applications of EEG-based robot control and navigation are undergoing a major transformation in dimension as well as scope. Under the background of artificial intelligence, medical and nonmedical robots have rapidly developed and have gradually been applied to enhance the quality of people’s lives. We focus on connecting the brain with a mobile home robot by translating brain signals to computer commands to build a brain-computer interface that may offer the promise of greatly enhancing the quality of life of disabled and able-bodied people by considerably improving their autonomy, mobility, and abilities. Several types of robots have been controlled using BCI systems to complete real-time simple and/or complicated tasks with high performances. In this paper, a new EEG-based intelligent teleoperation system was designed for a mobile wall-crawling cleaning robot. This robot uses crawler type instead of the traditional wheel type to be used for window or floor cleaning. For EEG-based system controlling the robot position to climb the wall and complete the tasks of cleaning, we extracted steady state visually evoked potential (SSVEP) from the collected electroencephalography (EEG) signal. The visual stimulation interface in the proposed SSVEP-based BCI was composed of four flicker pieces with different frequencies (e.g., 6 Hz, 7.5 Hz, 8.57 Hz, and 10 Hz). Seven subjects were able to smoothly control the movement directions of the cleaning robot by looking at the corresponding flicker using their brain activity. To solve the multiclass problem, thereby achieving the purpose of cleaning the wall within a short period, the canonical correlation analysis (CCA) classification algorithm had been used. Offline and online experiments were held to analyze/classify EEG signals and use them as real-time commands. The proposed system was efficient in the classification and control phases with an obtained accuracy of 89.92% and had an efficient response speed and timing with a bit rate of 22.23 bits/min. These results suggested that the proposed EEG-based clean robot system is promising for smart home control in terms of completing the tasks of cleaning the walls with efficiency, safety, and robustness.
The idea of interfacing brains with machines/robots has long captured the human imagination. Brain-computer interface (BCI) technology intend to build an interface between the brain and any electrical/electronic device (e.g., a wheelchair, smart home appliances, and robotic devices) using electroencephalogram (EEG) which is a noninvasive technique for measuring electrical potentials from electrodes placed on the scalp produced by brain activity. Nowadays, the EEG technique has been used to establish portable synchronous and asynchronous controls for BCI applications. Noninvasive EEG-based BCIs are the most promising interface for space of applications for people with severe motor disabilities because of their noninvasiveness, low cost, practicality, portability, and being easy to use. For some disabled patients with physical disability or paralysis while the brain function is still normal, although they have a normal large brain consciousness and thought, they cannot communicate with the external environment through the severely damaged muscle and nervous system and complete the daily work independently. This has caused serious physical and mental trauma, and their lives are very painful, which will affect their recovery process to some extent. How to restore or enhance their control and communication capabilities to the outside world has been the goal that has been pursued for many years in the field of medical rehabilitation. Therefore, BCIs can be used for helping patients with severe brain disorders or muscle damages to regain their ability to communicate directly with the outside environment through the brain electrophysiology response [1–3]. BCI can also be beneficial for the elderly as advanced assistive and rehabilitative technologies and useful for young able-bodied for controlling video games for entertainment [4, 5] or controlling a robotic arm for several purposes [6–9]. However, most of the traditional brain-computer interface equipment is expensive, bulky, and tedious, which makes it difficult to popularize and apply brain-computer interface technology in real life. The portable brain-computer interface has become one of the hotspots in the field of the brain-computer interface because of its advantages of easy to carry, easy to use, safe, and reliable.
BCI technology is mainly divided into two types of brain activity measurement, invasive BCI, and noninvasive BCI, depending on the way of putting the electrodes to record the electrical brain activity [10–14]. Among them, the invasive BCI might lead to an immune reaction, which causes serious harm to the user, and it is hardly accepted by disabled people because of the invasiveness of the technique which requires a dedicated surgery, and its cost with equipment is very expensive and not covered by many governments yet. Although the noninvasive brain-computer interface is less accurate than the invasive BCI, it is still relatively cheaper compared with all other techniques and everyone can easily accept it. There are several paradigms to control machine or computer using our brain signal characteristics and the most popular ones are motor imagery [15, 16], P300 wave [17, 18], steady state visual evoked potentials (SSVEP) [19–21] for building practical brain-computer interface systems. So far, the SSVEP method was applied widely because of the high signal-to-noise ratio and robustness . SSVEP induction means that when the human brain receives the stimulation of a fixed frequency scintillation block, an uninterrupted response related to the stimulation frequency will be generated in the visual cortex of the human brain. This SSVEP brain response is a very useful natural involuntary phenomenon which has been tested by researchers many times. The earliest SSVEP-BCI system, designed by Regan , in 1979, allowed subjects to select a flashing button on the computer screen by simply looking at the computer screen , basically achieving the desired design goals. Then, Mullerputz and Guneysu and Akin applied the SSVEP-BCI system to the physical control of neural limb and humanoid robot, respectively, and achieved good control results . In this paper, we chose SSVEP because it does not need any training phase for subjects and has very high accuracy compared with P300 or motor imagery using single trial electroencephalography (EEG) signal. The commonly used signal processing and classification methods of SSVEP include fast Fourier transform (FFT), wavelet transform, canonical correlation analysis (CCA), linear discriminant analysis (LDA), and support vector machine (SVM). In this paper, CCA was used for developing our signal processing algorithm. Compared with other SSVEP signal classification algorithms [10–14, 25], CCA classification algorithm is fast, efficient, simple, and easy to use.
In some previous researches, the SSVEP paradigm was successfully used in writing tasks . In the paper , we can see that the authors proposed a hybrid brain-computer interface system that combines P300 and SSVEP modalities. This combined system has improved the accuracy of EEG-based wheelchair control. In addition, SSVEP has been also used in the mental spelling system [28, 29]. In the paper , the authors used three flash speeds to control the small robot car. Lee et al. only use OZ as the reference electrode to collect and process EEG signals. In the paper , Lu and Bi have proposed a longitudinal control system for brain-controlled vehicles based on EEG signals. However, it is still unknown whether it can be used in the industry.
In this paper, a new type of intelligent crawler robot is designed for cleaning the walls, which is considered as one of the smart home appliances. Compared with the wheeled robot , the crawler robot has the advantages of long life and high carrying capacity. The intelligent crawling robot for the walls used in this experiment adds an adsorption device using negative vacuum pressure, which effectively solves the problem of sliding of the cleaning robot on a wall with a certain inclination angle. The BCI based on SSVEP can usually provide a high information transmission rate, the verification process of the system is relatively simple, and no training of the subjects is required. However, because the SSVEP of some subjects is very weak and vulnerable to the interference of other noise signals, how to accurately identify SSVEP from a short time window is still a challenging problem in BCI research based on SSVEP. This is also the subject that we will continue to study in the future. In this study, the SSVEP paradigm was designed to control the crawler robot for cleaning the dust on the walls. We used the high accuracy SSVEP paradigm to cooperate with our cleaning robot to complete the designed experiment. To our best knowledge, this is the first report, which used brain machine interface for crawling cleaning robot control to help persons with disabilities to improve their quality of life.
This paper is arranged as follows: in the Materials and Methods section, the experimental paradigm and analysis method of brain signal and the motion model of the intelligent crawling robot were introduced. At the same time, the offline experiment and online experiment are completed, and the data analysis is carried out. In the Results section, the offline and online experiments were summarized and discussed separately, and the accuracy and ITR of the experiment were obtained. Our experiments validate our views and achieve the desired results. In the Discussion part, we mainly talk about the limitations of the system and put forward the future changes. Finally, conclusion and prospects of future work are given in Section 5.
2. Materials and Methods
2.1. Participants and Experimental Paradigm
Seven healthy volunteers (4 males and 3 females, 23–27 years of age) were invited to join the experiment for performing some robot control tasks using their brain activity. None of the subjects have prior experience on brain-computer interfaces. Clear written informed consent was obtained from all the participants, who were informed in detail about the purpose and possible consequences of the experiment. The experimental protocol was carried out in accordance with the latest version of the Declaration of Helsinki.
The experiments were carried out in a quiet and comfortable environment to reduce the noise effect on our EEG recording. Subjects sat on a chair which is 60 cm away from the screen which contains the stimulation interface. In order to ensure the accuracy of the experiment, participants were required to avoid gnashing during the experiment. Because the SSVEP paradigm was easy to cause fatigue, the subjects can take a rest after one session. The flow of the experiment is shown in Figure 1. The experimental process is mainly divided into three parts. Firstly, the EEG acquisition device should be worn correctly for the subject and the subject’s position should be adjusted. Secondly, the collected data are processed and classified by a signal processing computer. Finally, the processed instructions are sent to the lower computer, that is, the intelligent crawling robot.